This section of the blog contains articles about the Oracle suite of Enterprise Content Management applications. This includes Universal Content Management (UCM), Web Content Management (WCM), Universal Records Management (URM), and a little bit of Information Rights Management (IRM). I helped create several of these products, and thus am very opinionated about how they should be used... I also cover technologies and topics relevant to content management in general, such as enterprise search, and identity management.
Besides the articles in this section, you may also benefit from the following sources:
Primary Oracle ECM Sources
- The Yahoo Group For Stellent Users: this is an email list with 2500 subscribers, and over 15000 messages in the archives
- The Oracle Enterprise Content Management Forum: a more "official" forum, although it gets less traffic, and has far fewer items in its archive
Oracle ECM Blogs
- Fusion ECM: the official Oracle blog on content management, starring Billy Cripe, and occasionally Raoul Miller.
- Oracle IRM: the official Oracle blog for Information Rights Management, starring Simon Thorpe.
- Kyle's Blog On UCM: another Oracle "Best Practices" blog, staring Kyle Hatlestad
- ECM Alerts: Oracle ECM product marketing blog, for official news about events and new product launches
- Content On Content Management: David Roe's blog on Oracle Content Management.
- webmonkeymagic: mikeyc7m's tips and rants on web content management, Site Studio, SSPU, and the like.
- John Sim's blog: web content management, and web design blog. Lots of Site Studio tips.
- Jason Stortz's blog: IdocScript, Components, Site Studio, and general tips and tricks.
- Ryan Sullivan's blog: the usual ECM stuff!
Additional Oracle ECM Resources
- Official Oracle Content Management Wiki: a good place to start to find information on content management.
- Main Oracle ECM Page: includes links to all the product lines (imaging, conversion, records management, and UCM).
- Official 10gr3 documentation: more than you'll ever need to know...
- Free Oracle ECM Samples Components.
- Independent Oracle Users' Group: a small handful of Stellent resources, including a listserv.
- SCS Components: an site with some free and commercial components for the content server.
- Recorded Oracle Events: general recording of Oracle webcasts, including a few about Stellent.
- RSS Feed for Yahoo Group: For some reason, Google won't spider Yahoo's site very well, so I publish this RSS Feed to make content more findable.
If you know of another notable Oracle ECM site, send me an email! I'd define "notable" as any "official" site, or a site that posts useful information at least once per month...
Another talk I gave at Collaborate 2013 is this one on ADF Mobile and WebCenter. It builds off my talk from last year about general techniques, and gets into specific about the new ADF Mobile technology, and how to integrate it with WebCenter content and WebCenter Portal.
At Collaborate 2013 this year, Tony Field and I put together a talk about a topic that has been been floating around the WebCenter community as of late...How do I integrate WebCenter Sites (Fatwire) with WebCenter Content or Site Studio? We put together a handful of integration techniques, but the main focus was on upcoming features in the next version of WebCenter... specifically the official Sites/Content connector, and support for External Repositories. Cool by themselves, but when combined with Site Studio for External Applications, it's a compelling set of integration options:
How would you like to leave Collaborate knowing exactly what you wanted to learn? Here's your chance...
Like last year, the WebCenter SIG at IOUG Collaborate 2013 (April 7-11 in Denver) will have a deep dive session for Sunday. Bezzotech was asked to deliver 2-hours of a deep dive... and were batting around ideas for what to talk about... Security? Performance? Integrations?
Then it hit us, why not let the attendees pick our talk?
If you always wanted to know something crazy about how WebCenter works, please take our survey so we know what to present. You can also leave a comment, email us at firstname.lastname@example.org, or send it to me directly. We'll tally up the requests and let the WebCenter faithful decide what our talk will be!
I'm genuinely curious about what you are curious about ;-)
I was recently doing some training on ADF, and the students were complaining how slow JDeveloper was... Dragging and dropping Data Controls onto a JSF page? It's the pause of death if you will. Not to mention the "Out Of Memory" errors that crop up in the middle of debugging a large app. Very frustrating for developers, so I decided to once and for all get figure out what magic JVM tuning parameters would speed it up.
As a general rule, Java is optimized for throughput, not latency. Once the garbage collector kicks in, performance drops like a rock. A 2 second pause every once in a while is OK for a server, but for an IDE it's misery. So here's the fix:
- Go to your JDeveloper root directory, is should be something like C:\Oracle\jdev\Middleware\jdeveloper
- Open the file ide\bin\ide.conf, scroll down to the default memory settings:
- Boost the memory to something larger, like so:
- Open the file jdev\bin\jdev.conf
- Add the following config settings:
- Then restart JDeveloper... If it doesn't start, you'll need to reduce the amount of memory allocate in the ide.conf file from step 3.
AddVMOption -Xms128M AddVMOption -Xmx768M
AddVMOption -Xms1024M AddVMOption -Xmx1024M
# optimize the JVM for strings / text editing AddVMOption -XX:+UseStringCache AddVMOption -XX:+OptimizeStringConcat AddVMOption -XX:+UseCompressedStrings # if on a 64-bit system, but using less than 32 GB RAM, this reduces object pointer memory size AddVMOption -XX:+UseCompressedOops # use an aggressive garbage collector (constant small collections) AddVMOption -XX:+AggressiveOpts # for multi-core machines, use multiple threads to create objects and reduce pause times AddVMOption -XX:+UseConcMarkSweepGC
And that's it! Your mileage may vary, of course... And you may need additional parameters, depending on what version of JDeveloper you're running. Just keep in mind that you are tuning Java for shorter pauses, and not greater throughput.
UPDATE 1: some students still had issues, so in addition to the JVM settings, I've found these tips also help out:
Go to Tools / Preferences / Environment, and switch to the "Windows" look and feel. The Oracle look and feel is prettier, but slower.
Disable all extensions that you don't need. This is usually a huge savings... Go to Tools / Preferences / Extensions, and turn off thnigs you know you don't need. One thing I do is disable all extensions by default, then enable only the ones I know I need for my current project. For example, disable everything, then enable only those extensions that start with ADF. This will automatically enable dependent extensions. Enable others (Portal, SOA, RIDC) only if needed.
Open all documents in "Source" mode by default. Go to Tools / Preferences / File Types, and click the Default Editor tab. For all web pages (HTML, JSF, JSP) set the default editor to "Source". You can always click the "Design" tab to see the design. For best results, select items in the "Structure" window (by default on lower left) and edit them in the "Property Inspector" window (by default on the lower right).
If you really want to get extreme... you can install a solid-state hard drive for your workstation. Barring that, if you have enough RAM you can allocate 4 GB and create a RAM driver for your system. This looks like a normal hard drive, but it's all in RAM. Then install JDeveloper on that, and it will be almost as good as a solid state drive.
Other developers have had success using
UPDATE 2: A reader has informed me that this line:
Breaks offline database support in JDeveloper... so that one will have to be avoided in some cases.
I was never really a fan of mobile applications: I prefer the mobile web experience. Every mobile device supports HTML5, which means that you can do just about everything a mobile app can do, other than high-performance graphics. In fact, according to an Adobe study users prefer mobile web to mobile apps for just about everything.
It covers a lot of turf, but there are four key takeaways:
- You will need to learn HTML5 eventually, no matter what
- Your mobile strategy should default to the mobile web: mobile apps are rarely needed, and add unnecessary complexity
- And please make sure your mobile strategy is a natural extension of your business model... or you'll be out a lot of money!
Hope this helps!
I'm continuing my tradition of doing my blog year-in-review in late April... mainly because I started my blog six years ago on April 29th. But, also in the hopes it would stand out more, since everybody else has a fiscal-blog-year-end on December 31st!
In the 2011/2012 time frame I had 204,514 page views, which is a 12% increase over the previous year! Woah... surprised to see that spike, considering I've been feeling guilty about not posting enough these days... altho a lot of that was because my post on how Steve Jobs couldn't program a computer was on the front page of Hacker News for a few days! The top posts from 2011 are as follows:
- Oracle Acquires FatWire! And people are curious as to what will happen next...
- Oracle Mix Jumped The Shark: a rant against how some folks rigged the Mix system to con their way into getting Open World Sessions. Uncool.
- WebCenter Performance Tuning: a case study in WebCenter Content tuning
- WebCenter 11g PatchSet5: some useful downloads and links
- One WebCenter To Rule Them All: some coverage of the rebranding that occurred at OpenWorld 2011
- Collaborate 2011 Presentations: always good stuff there ;-)
- Mashup Standards: JSON-P Versus CORS! My opinions on why JSON-P is superior, and an add-on to my popular jQuery Plugin to better support mashups
- PowerPoint Tips From South Park: the title says it all...
- More On FatWire: additional analysis and predictions
- Multilingual UCM: some info on how to support multiple languages with UCM
This year I should have time to add more tutorials... there's some nifty stuff coming out in the next version of WebCenter that could change how people do enterprise integrations. Stay tuned...
Oracle recently acquired FatWire, and renamed it WebCenter Sites. It is a "web experience management" toolkit, which is similar to Oracle's existing Site Studio product -- a part of Oracle UCM, now called WebCenter Content.
After using Site Studio for years, I got pretty accustomed to it's terminology and toolkits... so looking at FatWire was initially intimidating because it was just so dang different. But, after using it for several months, I've come to the conclusion that a lot of the fundamentals are pretty similar. Pretty much everything Site Studio does is built in to FatWire, and FatWire has a few nifty extras as well.
So, for IOUG Collaborate this year, I put my insights together into a presentation: Crash Course in FatWire for Site Studio Developers:
It's not a replacement for actual training... but it does cover all the major low-level assets, and how they fit together to form a site. If you know a thing or two about Site Studio, this should help you get over the initial "fear of the unknown!"
Oracle UCM Patch Set 5 is released! And thus begins the long, long hunt for the patches you need to upgrade...
Most of this info is available in the Oracle FAQ for ECM 11g, as well as the Oracle Fusion Middleware Patching Guide... but I was tired of it being un-googleable... So I decided to put a few of the links together here.
Upgrading ECM can be a multi-step process. You need to upgrade WebLogic before upgrading ECM, and you need to make sure you have the right version of the Repository Creation Utility (RCU)... not to mention the multi-gigabyte general installer for ECM itself (which includes IPM, UCM, IRM, and URM). If it's a new install, just grab the most recent Weblogic Server downloads. Otherwise, use the upgrade installers below:
Patch Set 3
- Patch 11061000: FMW 11G PS3 (188.8.131.52) FOR ECM
- Patch 11060956: FMW 11G PS3 (184.108.40.206) FOR RCU
- Patch 11060985: Oracle WebLogic Server11gR1 UPGRADE installer (10.3.4)
Patch Set 4
- Patch 12395130: FMW 11G PS4 (220.127.116.11) for ECM
- Patch 12395110: FMW 11G PS4 (18.104.22.168) for RCU
- Patch 12395574: Oracle WebLogic Server11gR1 UPGRADE installer (10.3.5)
Patch Set 5
And yes, in case you noticed, I'm using those nifty short URL for Oracle patches I set up... makes the URLs a lot easier to digest, don't ya think?
How many times has this happened to you???
You're looking around Oracle for the latest patches, and after copious amounts of digging, you finally find the mystery patch that you need... you click on the "download" link, install it, and you're good to go!
Later on... your client, or co-worker, or somebody on the message board asks, "How'd you do that?" And because you have a photographic memory, you reply "With patch 12395560, of course!" Then they ask, "got a link?" And then you say this:
https://support.oracle.com/CSP/ui/flash.html#tab=PatchHomePage(page=PatchHomePa ge&id=gj46o799()),(page=PatchSearchResultsHome&id=gj46pr1y(search=%3CSearch%3E% 0A%20%20%3CFilter%20name=%22patch_number%22%20op=%22IS%22%20value=%2212395560%2 2%20type=%22patch_number%22/%3E%0A%20%20%3CFilter%20name=%22platform%22%20op=%2 2IS%22%20value=%22%22%20type=%22platform%22/%3E%0A%3C/Search%3E&incFamilyProds= false&flag=search))
Yikes... not exactly 'twitter friendly.'
In order to simplify the process (and make my documentation more readable), I set up a URL Shortener for Oracle patches for myself. Unlike most URL shorteners, it takes a parameter. The number after the slash is the Oracle patch number... which should be easy to spot on the form. So, instead of the crazy URL above, you could use one of these two:
The first one goes to the standard My Oracle Support page -- with all it's flashy goodness -- and gets as close to a "quick-link" that I could deduce. The second URL goes to the old fashioned Oracle Updates web site, which supports parameterized URL quite nicely. Guess which one I prefer? ;-)
Ideally, the Oracle support team would implement a parameter-based redirect themselves... and expose that "quick link" on the support page. Until then, I'm going to do it this way. I wonder if it will catch on???
For your consideration...
- Oracle Patches:
- Oracle Bugs:
In part 1 of this post, I covered the JSON-P "standard" for mashups. Not so much a standard per se, but a sneaky way to share JSON code between servers by wrapping them in a 'callback' function... For example, if we have our raw JSON data at this URL:
A direct access would return the raw data dump in JSON format:
Since JSON-P is something of a hack, many developers started looking for a more secure standard for sharing JSON and XML resources between web sites. They came up with Cross-Origin Resource Sharing, or CORS for short. Enabling CORS is as simple as passing this HTTP header in your XML/JSON resources:
Then, any website on the planet would be able to access your XML/JSON resources using the standard XmlHttpRequest object for AJAX. Despite the fact that I like where CORS is going, and see it as the future, I just cannot recommend CORS at this point.
Since CORS is built on top of the XmlHttpRequest object, it has much nicer error handling. If the server is down, you can recover from the error and display a message to the user immediately. If you use JSON-P, you can't access the HTTP error code... so you have to roll-your-own error handling. Also, since CORS is a standard, it's pretty easy to just put a HTTP header in all your responses to enable it.
My big problem with CORS comes from the fact that it just doesn't seem that well supported yet... Only modern browsers understand it, and cross-domain authentication seems to be a bit broken everywhere. If you wanted to get secure or personalized JSON on a mashup, your back-end applications will need to also set this HTTP header:
And, in theory, the AJAX request will pass along your credentials, and get back personalized data. The 1.7 jQuery plug-ins works well with JSON-P and authentication, but chokes badly on CORS. Also, keep in mind that authenticated CORS is a royal pain in Internet Explorer. Your end users will have to lower their security setting for the entire mashup application in order to make authenticated requests.
Now, JSON-P isn't great with security, either. Whereas CORS is too restrictive, JSON-P is too permissive. If you enable JSON-P, then you pass auth credentials to the back-end server with every request. This may not be a concern for public content, but if an evil web site can trick you into going to their mashup instead of your normal mashup, they can steal information with your credentials. This is call Cross-Site Request Forgery, and is a a general security problem with Web 2.0 applications... and JSON-P is one more way to take advantage of any security holes you may have.
In addition, the whole CORS process seems a bit 'chatty.' Whereas JSON-P requires one HTTP request to get secure data, CORS requires three requests. For example, assume we had two CORS enabled applications (app1 and app2) and we'd like to blend the data together on a mashup. Here's the process for connecting to app1 via CORS and AJAX:
- Pre-Flight Request: round-trip from client browser to app1 as a HTTP 'OPTIONS' request, to see if CORS is enabled between mashup and app1
- Request: if CORS is enabled, the browser then sends a request to app1, which sends back an 'access denied' response.
- Authenticated Request: if cross-origin authentication is enabled, data is sent a third time, along with the proper auth headers, and hopefully a real response comes back!
That's three HTTP requests for CORS compared to one by JSON-P. Also, there's a lot of magic in step 3: will it send back all the auth headers? What about cookies? There are ways to speed up the process, including a whole ton of good ideas for CORS extensions, but these appear to be currently unpopular.
Conclusion: Use JSON-P With Seatbelts
If all you care about is public content, then CORS will work fine. Also, it's a 5-minute configuration setting on your web server... so it's a breeze to turn on and let your users create mashups at their leisure. If you don't create the mashups yourself, this is sufficient.
However... if you wish to do anything remotely interesting or complex, JSON-P has much more power, and fewer restrictions. But, for security reasons, on the server side I'd recommend a few safety features:
- Validate the HTTP_REFERER: only allow JSON-P requests from trusted mashup servers, to minimize request forgery.
- Make JSON-P requests read-only: don't allow create/modify/delete through JSON-P.
But wait, isn't it easy to spoof the HTTP referrer? Yes, an evil client can spoof the value of the referrer, but not an evil server. In order for an evil mashup to spoof the referer, he'd have to trick the innocent user to download and run a signed Applet , or something similar. This is a typical trojan horse attack, and if you fall for it, you got bigger problems that fancy AJAX attack vectors... DNS rebinding is much more dangerous, and is possible with any AJAX application: regardless of JSON-P or CORS support.
Links and Free Downloads
For those of you interested in Oracle WebCenter, I created a CrossDomainJson component that enables both CORS and JSON-P, and it includes some sample code and documentation for how to use it. It currently works with WebCenter Content, but I might expand it to include WebCenter Spaces, if I see any interest.
For those of you in the Toronto area, I'll be presenting at the AIIM/Oracle Social Business Seminar this Thursday! Its at Ruth's Chris Steakhouse, 145 Richmond Street West, Toronto, ON. The agenda is as follows:
- 10:00 a.m: How Social Business Is Driving Innovation, Presented by: John Mancini, AIIM
- 11:00 a.m: Solving the Innovation Challenge with Oracle WebCenter, Presented by: Howard Beader, Oracle
- 12:00 noon: Lunch and Networking, Table Discussions on Case Study Challenges
- 1:00 p.m: Strategies for Success Case Study, Presented by Bex Huff, Bezzotech
- 1:45 p.m: Final Remarks
Space is limited, so register now for a seat!
In my previous post, I was talking about the JSON-P standard for mashups. It's very handy, but more of a "convention" than a true standard... Nevertheless, it's very popular, including support in jQuery and Twitter. In this post I'm going to discuss what some consider to be the modern alternative to JSON-P: Cross-Origin Resource Sharing, or CORS for short.
Lets say you had two applications, running at app1.example.com and app2.example.com. They both support AJAX requests, but of course, they are limited to the "Same-Origin Policy." This means app1 can make AJAX requests to app1, but not to app2. Let's further assume that you'd like to make a mashup of these two app at mashup.example.com.
No problem! In order to enable cross-origin AJAX, you simply need to make sure app1 and app2 send back AJAX requests with this HTTP header:
This is easily done, by adding one line to the Apache httpd.conf file on app1 and app2:
Not to mention, because the XmlHttpObject is used, CORS has much better error handling than JSON-P. If there's an error accessing a file, you can catch that error, and warn the end user. Contract that with JSON-P, where there's no built-in way to know when you can't access a file. You can build your own error handling, but there's no standard.
Nevertheless, I still prefer JSON-P for mashups. Why? Well, it boils down to two things: performance, and security. I'll be covering the specifics in part 3 of this port.
In a recent project, I had a client who wanted to resurface Oracle UCM content on another web page. The normal process would be to use some back-end technology -- like SOAP, CIS, or RIDC -- to make the connection. But, as a lark, I thought it would be more fun to do this purely as a mashup. I would need to tweak UCM to be more "mashup-friendly" -- I'll be sharing the code (eventually) -- but first I needed to do some research on the best mashup "standard" out there.
You would then use the standard AJAX XmlHttpResponse object, parse this JSON data, then do something with the message. My jQuery Plugin for UCM does exactly this... but of course has the limitation that it will only work on HTML pages served up by UCM. You can use fancy proxies to bypass this limitation, but it's a pain.
Instead, if UCM supported 'padded JSON', the process would be different. The URL would look something like this:
In this case, the callback=processData parameter triggers the server to 'wrap' the JSON response into a call to the function processData. Then, instead of using the XmlHttpResponse object, you'd use good old-fashioned remote scripting. Like so:
Now... JSONP is a good idea, but it's about 5 years old... A lot of newer browsers support a slightly different standard: Cross-Origin Resource Sharing. It's an actual standard, unlike JSON-P which is more of a convention... the purpose is to safely allow some site to violate the silly "Same-Origin Policy". I'll be covering CORS in part 2 of this post, including the security enhancement. But, in part 3 I'll explain why I still prefer JSON-P, provided you add some extra security.
I gave two presentations at Oracle Open World this month... one on Integrating WebCenter Content: Five Tips to Try, and Five Traps to Avoid! I broke it down into the big sections: contribution, consumption, metadata, security, and integrations. Special thanks to IOUG for sponsoring this talk!
My second talk was a case study based on a big project that completed recently, integrating WebLogic Portal, UCM, E-Business Suite, Autonomy IDOL, and a whole bunch of other stuff to make a global e-commerce web site. The client is in a highly regulated industry, and I was unable to get permission to use their name... but if you're curious about the details ping me!
If I missed you at Open World, I hope to see you at IOUG Collaborate 2012!
The WebCenter Portal team has put together a VirtualBox virtual machine to showcase the WebCenter Portal product. You can download it from Oracle. It's a big one: clocking in at 30 GB, so pack a lunch before downloading it.
The install instructions are pretty good for Windows and Linux clients... but if you're on a Mac (like me), it's missing one important tip. The file REAVDD-HOL-WC.ovf contains the information needed to import the files into a VirtualBox VM... but if you're running the free version of VirtualBox, it chokes on the import every time. The culprit is this line:
If you're on Windows, and have a D drive, this works fine... but if you're on a Mac (and probably Linux), this will break the import. The fix? Use this XML instead:
And re-do the import... you'll need to re-set-up sharing once it's running. But at least now it will have a valid path!
NOTE: This is just meant to be a sandbox for testing integrations, and the like. It's not meant to be placed into a production environment... but, like all demo code, I'm sure I'l find it floating around in production eventually... and have to make it work.
I was always a bit little skeptical about the initial mobile offerings for UCM and WebCenter. They never impressed me, because I felt strongly that these apps were fundamentally flawed in their design...
Why? Because they focused on being Mobile Applications instead of Mobile Web. The first time I held an iPhone, I noticed that it was running a browser that supported HTML5. The first Android was the same. This was at a time where HTML5 support was rare on desktop browsers, and few developers knew how to use it. Nevertheless, I predicted years ago that it would be the future... HTML5 was so powerful, that Flash and native mobile apps were unnecessary for 95% of applications. Many clients asked my advice on mobile apps, and my answer was always the same: "Skip native apps, and focus on the mobile web!"
This week, Oracle announced their next generation of the ADF Mobile toolkit... and (as I predicted) they are going the same route! Native code is no longer the focus: previously, you would create an ADF component, and it would be compiled down into native iOS or Android controls. No more! The next version will compile to HTML5 and be rendered in the mobile browser!
If you prefer to roll-your-own UI, I'd recommend Zepto as a minimalist framework instead...
What's next for the web, then? I believe that mobile application development will be the biggest driver for the adoption of HTML5 browsers. Yes, probably only 10% of mobile phones are HTML5-enabled smart phones... but people cycle through cell phones every 2 years. Compared that to the enterprise, some of which stubbornly refuse to upgrade from IE6!
I'd bet 90% of Americans will have a HTML5 mobile phone, before 90% of them are off IE6! Sad, but true... but good news for mobile developers!
UPDATE: Dang it! Just as soon as I blog about this, Adobe goes and purchases PhoneGap! What does this mean for Oracle? Tough to say... it's probably a good thing, since most of PhoneGap is open source. The only piece that's not Open Source is their nifty build engine. But, since Oracle already owns their own build engines (jDeveloper and Eclipse plugin), this is not a stumbling block.
UPDATE 2: It appears that Adobe has done "The Right Thing" and is submitting PhoneGap to the Apache group, and re-branding it as Project Callback. This will hep cement it as "the standard" toolkit for mobile app development.
I love doing performance tuning... It's typically a mundane process of tiny tweaks and digging for gold in log files, but for some reason I find it a blast. I usually do it for every client, and sometimes I have projects dedicated exclusively to tuning.
One cropped up recently, where a client was having craaaaazy slow performance with an 11g custom component that Shall Remain Nameless. It worked fine with small data sets, but on a page with 1000 items, the load time was 12 minutes. Thus begins our adventure! But first, some wise words from the grandfather of performance optimization:
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil" -- Donald Knuth
Yarp... The first thing to do is establish a benchmark. Run some kind of automated test to time performance, and then start your tweaking. Also, be sure to put it in a nice table, so you can visually see just how awful the starting point is!
It's kind of a dull process-of-elimination to disable components, restart, and run another benchmark... but you have to do that in order to narrow down the offending code. I initially thought the problem was in one component... I was wrong... So, as boring as it is, it must be done to eliminate red herrings.
Once I found the offending component, I turned on some low-level tracing parameters. You can do this from the System Audit Information administration page. I usually turn on Verbose tracing, as well as these sections: requestaudit, pagecreation. When I did a small manual test, what I got back shocked me a bit:
pagecreation/6 09.13 17:55:47.774 IdcServer-31333 page generation took 10766 ms; gets 58863 funcs 24362 incs 9256 eval 160 requestaudit/6 09.13 17:55:48.012 IdcServer-31333 FOO_SERVICE 14.5684232711792(secs)
Woah... even a simple page was taking 14 seconds, 10 of which was in rendering the IdocScript! I checked, and these pages were far too complex for words. On a page with a 1000-item list, the component that Shall Remain Nameless was spitting out 15 Megs of HTML! Danger! Danger!
Problem solved, right? Not so fast... even with that the site was still slow. Next I suspected something in the database: bad histogram, missing index, whatever. So I turned on another low-level flag: systemdatabase. Please note: enabling this will turn your log files into a frigging Russian novel, so don't leave it on for too long... I ran another test, and got something like this:
systemdatabase/6 09.14 15:13:36.548 IdcServer-18413 53 ms. SELECT FOO_QUERY [Executed. Returned row(s): true] systemdatabase/6 09.14 15:13:58.803 IdcServer-18413 Closing active result set systemdatabase/6 09.14 15:13:58.804 IdcServer-18413 Closing statement in closing internals
Well... I was wrong again! Can you see the issue here? The query only took 53 milliseconds to run, so the database is hunky dory! But, it takes a full 12 seconds for UCM to release the connection. That means, somewhere deep inside the component that Shall Remain Nameless, some Java code is taking 12 seconds to post-process the database results. In some scenarios, it took over 40 seconds just to post-process the results.
What the heck is it doing???????
Don't know, don't care! Instead of digging through the component's code and trying to optimize that 40 second delay, I did what any sane web developer should do when faced with a poor performing black box: make a data cache! After quite a bit of digging, I found the magic function that was so very important that it took 40 seconds to run. So instead of calling it directly, I wrapped it up in a secure data cache. That cut the 40,000 milliseconds down to 4. You sometimes see stale data on the page, but that's always the case with a web interface... so it's usually a good tradeoff.
So... after improving their performance by 5 orders of magnitude, I patted myself on the back, and flew home...
The final benchmark numbers looked amazing... and I think that's why I find performance tuning is so satisfying. It's so very lovely to see the difference between the initial and final performance... and it's even better if you need to use a logarithmic scale to see them! ;-)
Open World is barely a month away! I'll be heading there early for some Oracle ACE briefings and the like... I'm normally a "broadcast only" Twitter user, but when I'm at conferences I check it all the time, and tweet with location services on. If you want to meet up, just message me!
I have a couple of sessions this year... unfortunately they are all on Thursday! Dang it! I was hoping to leave the conference early -- since Michelle and I are having our first kid, and her due date is a few weeks after Open World. Alas, the scheduling gods were not with me:
- Session: 10843
- Creating a Global E-Commerce Site with Oracle E-Business Suite and Oracle Fusion Middleware
- Thursday, 12:00 PM, Intercontinental - Intercontinental Ballroom B
- Session: 9539
- Integrating ECM into Your Enterprise: 5 Techniques to Try and 5 Traps to Avoid
- Thursday, 03:00 PM, Intercontinental - Telegraph Hill
I know picking Open World sessions can be a bit of a baffling ordeal... so if you're pressed for time, I'll suggest a few tips. If you want to see WebCenter based content, check out the WebCenter partner sessions. Lots of good stuff there. If you're curious about non-WebCenter products but don't know where to start, I'd recommend the Oracle ACE sessions over just about everything else. ACE sessions are a good bet: speakers are usually very knowledgeable, very passionate, and very excited to share what they know. Translation: minimal marketing fluff. You don't get the title "Oracle ACE" by being a self-promoting fool!
Well... at least most of the time Oracle ACE's aren't self-promoting fools... there are exceptions.
Jump The Shark: (verb) a term to describe a moment when something that was once great has reached a point where it will now decline in quality and popularity.
Oracle MIX is a social software app to connect people in the Oracle universe. It was launched back in 2007 by The Apps Lab so people could network and stay connected during (and after) Open World. It was at the time the largest JRuby on Rails site out there. It's a decent site, and you Oracle monkeys should check it out...
I believe in 2008, they decided to try something new: allow the community to "suggest a session" for Open World. They had ten slots at Open World, and everybody was encouraged to submit a session for consideration, and vote on what they liked. The ten sessions with the most votes would get to present at Open World.
This was also a great idea... It was the ideal place for sessions outside the mainstream to get a voice at Open World... technology that might be too "bleeding edge" for a general audience, but is the bread-and-butter of geeks who only hit one conference per year. Social software, mashups, open source, installing Oracle on a Roomba... you get the idea. If you want to do a mainstream talk about a mainstream product, then submit it through the normal channels to the Open World committee... If your session isn't picked, then it probably wasn't good enough.
This model worked fine in 2008, 2009, and 2010... but I think something went really REALLY haywire this year...
MIX, being an open community, allowed people to take the voting data and mash it up in interesting ways... Greg Rahn over at Structured Data did exactly this, and presented his data analysis of the votes. Just looking at the data I saw a lot of anomalies, but to me the smoking gun is this:
- Number of users who voted for exactly one author: 828
- Number of users who voted for ALL sessions by EXACTLY one author: 826
Well, that ain't right... once you dig further, you see what probably happened: the Oracle MIX community has been invaded by a spammer...
Specifically... somebody out there has a mailing list with a few hundred people, and contacted them all asking for votes. Probably repeatedly. I don't know about others in the MIX community, but I personally got three such emails begging for votes... One of them was so shady it probably violated Oracle's Single-Sign-On policy. The line between self-promotion and SPAM is fuzzy... but it was clearly crossed by a lot of people this year.
I know what you're thinking... must be sour grapes, eh? But no, I did not submit a MIX session. Oracle was kind enough to approve both of my Open World presentations this year, so I thought the gracious thing to do would be to leave the MIX sessions for the community... so I'm very disappointed in the behavior of these people.
The rules as-is are broken... based on Greg's data, 200 people at Microsoft could all vote for sessions like "Reason #6734 Why Microsoft Rocks and Oracle People are Big Fat Stupid Heads"... and they'd win every slot.
All communities have this problem... once they become popular, they become valuable. Once they become valuable, some people try to extract more value than their fare share. Many large sites implement some form of moderation or karma points to keep cheating to a minimum... I think it's about time MIX did the same. I have a few ideas for "guidelines":
- promotion via tweets and blogs is allowed and encouraged
- mass communication via emails or social networks will be considered "social spam," and grounds for disqualification
- "down-voting" like Digg should be enabled to further prevent spammers from carpet-bombing their way to the top
- sessions should be outside of mainstream Oracle talks: sessions similar to ones given at Open World are discouraged
- a maximum of two talks can be submitted on behalf of an individual, organization, or community group
- a maximum of one talk can be selected on behalf of an individual, organization, or community group
Of course, this isn't perfect... the top 10 slots could still go to people with 1000 employees, and therefore 1000 reliable votes! Probably the ideal situation is to randomly select some Oracle ACEs to be the judges every year, based on community input. Not ideal, but really hard to rig...
So... how many of you feel like you were "spammed" this year?
UPDATE: Oracle is soliciting opinions for what worked and what didn't this year. If you have an opinion about what should be fixed, please leave a comment on their blog or contact Tim Bonnemann directly.