This is why you're fat.

Yesterday, I watched the Oracle + Sun Strategy Update Webcast. It was certainly a surprising event -- surprisingly positive for the future of Sun and Java/JVM technologies, in my view. The promises Larry Ellison made at last year's JavaOne have been effectively redoubled in a way I myself could not have expected. The plan and expectation really is that Sun will quickly become a very profitable group within Oracle. And Java technologies will receive significant new investment -- not because investment in Java is expected directly to earn Oracle profits, but because the real money from Java is made through middleware sales, and in order for Oracle's middleware products to remain top notch, highly-scalable, state of the art, etc., the Java/JVM platform must move forward. Stagnation of the Java platform is not an option.

These are some of the things I heard while watching the webcast. For those of you who didn't have the opportunity to view the webcast, see my little Sun + Oracle Strategy Links (from Twitter) post.

I tweeted during the webcast (isn't that the only way to cover live technology events today?). Here are my tweets from the webcast (in time-forward order):

• Oracle + Sun webcast: a big focus on "We're hiring!" - across the board: sales, Solaris development, engineering new chips...
• "we're hiring!" -- the perfect statement for immediately building confidence among both customers and employees.
• JavaOne will become a global, travelling conference: U.S., Brazil, India, ...
• Oracle/Sun will invest in the Java developer community as well as specific Java technologies
• Java SE 7 goals: multiple languages, mulit-core support. Oracle JRockit will be applied/integrated with HotSpot.
• evolve Java EE 6: profiles important for customization; GlassFish will continue to be the Java EE reference implementation
• WebLogic will continue to be the high-end app server for enterprise sales
• Java ME: unify ME APIs with Java SE APIs; runtime optimization; optimize power consumption; focus on ME operability on different platforms
• different JVM models based on platform (enterprise, ME, etc.)
• significant investment in JavaFX; focus on designers; fusion of DHTML, JavaScript, Java, JavaFX capabilities within browser environment
• NetBeans continues as Lightweight IDE: focus on Java EE, ME, scripting; mobile development; reference implementation IDE for Java EE
• Hudson makes the presentation; will also be integrated with JDeveloper; Eclipse also remains important
• EE / GF enhancement: microkernel support; non-blocking I/O; Metro; lightweight, rapid development and deployment
• WebLogic: remains focus for enterprises; WebLogic and GF will share technologies. SOA: Open ESB will continue to be supported, among others
• OpenSSO will see continued investment; OpenDS will be maintained.
• speaking of Sun/Oracle, I'm right now remotely logged into a Solaris machine enhancing an app whose data flow is managed by an Oracle db
• Oracle will hire 2000 people, and only about 1000 will be laid off, as part of the merger consolidation
• arry highlights tape archive/backup as being very important (it is in the data center I work in, certainly)
• Forrester: can Oracle make more money from Java? Larry: where revenue comes from is less important than the fact that $ is made
• Larry: Java makes money mostly through sales of middleware. Hence, it's critical to continue to develop and enhance Java.
• Oracle's presence in the phone / devices realm will be the Java software that phones/devices run on
• Larry humorous on the "cloud computing" phrase -- today everything is "cloud computing" so how can you possibly be against it? I agree!
• Retweet from oracletechnet: Ellison: The only thing new about cloud computing is the word "cloud" - it's just a computer attached to the Internet
• Oracle sure is punctual: a major 5-hour event starts and ends within a few minutes of the publicized times!

Also, developers should take a look at the new Sun Oracle Overview and Frequently Asked Questions for the Developer Community page, which OTN (Oracle Technology Network) leader Justin Kestelyn (@oracletechnet) pointed me to last night.

In Java Today, Justin Kestelyn of the Oracle Technology Network pointed me to the new Sun and Oracle: Overview and Frequently Asked Questions for the Developer Community page:

Oracle has finalized the Sun transaction and the deal has closed. The combination of Oracle and Sun transforms the IT industry and will provide significant benefits and opportunities for the developer communities of the combined companies. For example, the combination of the Sun Developer Network (including java.sun.com), BigAdmin, and the Oracle Technology Network will result in the largest, and most diverse, community of Developers, Database Administrators, SysAdmins, and Architects. The richness and diversity of these communities will truly be remarkable. We know that you have many questions, and some of them we can answer now. We're also committed to providing updates regularly as more information becomes available. Note that the FAQ below is designed around developer community continuity specifically...

JavaFX expert Jim Weaver also watched the "Sun + Oracle Strategy" webcast, and posted notes and commentary in Oracle/Sun Strategy: We will invest heavily in JavaFX:

Today was a busy interesting day for lots of my fellow geeks (like you, perhaps) in that I found myself listening to two webcasts at the same time: The Apple iPad announcement, and the Oracle/Sun Strategy announcements.  Being a JavaFX developer, I was particularly interested in the extent to which Oracle is going to embrace JavaFX (and how much crow I would have to eat from my April 21, 2009 What, Me Worry post)...

Toni Epple did something I wish I'd thought of as I watched today's Oracle + Sun Strategy Update Webcast -- he used his system's screen capture facility! His Oracle & NetBeans post includes the slide about NetBeans, JDeveloper, and Eclipse:

Here’s a slide from the (currently ongoing) webcast about the Oracle - Sun Merger ... Good News, whatever it means in detail...

In today's Weblogs, Fabrizio Giudici asks And where have all the owls flown?:

I love owls; they are so elegant and have a strong personality (too bad in so many years I've been unable to take photos at any of them!). Unfortunately in my culture (and probably many others) they have also got a bad reputation, as they were considered messengers of bad fortune. It's for this reason that in italian a "gufo" is also a person who, for innate attitude or purportedly, makes always bad predictions - a doomsayer, in other words. In italian there's even a colloquial verb, "gufare" (literally "to act as an owl"), expressing this attitude. Since when Larry Ellison announced to the world that Oracle was buying sun, flocks of gufi (pronounce as the Disney character Goofy) exercised their pessimistic views on the fate of Sun, its products and its personnel. Recalling them in no particular order...

I gathered a few interesting Sun + Oracle Strategy Links (from Twitter):

For those of you who weren't able to spend 5 hours watching or listening to Wednesday's Oracle + Sun Strategy Update Webcast, here are a few post-event Twitter tweets from people I (@kevin_farnham) follow, that provide links to resources related to the webcast and its content: oracletechnet If you missed the Webcast today, here are some choice bits on demand: Charles Phillips and Larry Ellison http://bit.ly/cPOuSL #oraclesun...

John Ferguson Smart previews the Coding Dojo in Wellington next week:

We will be running the next Wellington Coding Dojo on February 2, 2010 - thanks again to the Wellington Java Users Group for helping to organize this session. The session will be held at Equinox (Equinox house, 111 The Terrace, Wellington) at 5:15. The interesting thing about these Coding Dojos is that they are not necessarily done on simple, abstract programming problems - we generally work with real (well, sort of ;-) ) working applications. The idea is to get a feel for TDD practices in a real-world situation, where it is useful to build a solution incrementally...

In the Forums, aragollum has an LWUIT question, Thread: Is there a way to implement a touch-sensible scrollbar?: I'm working on a project for N97, where scrolling is done by pressing on the scrollbar itself and moving it. Is there a way to implement this kind of behavior in LWUIT? It might be very helpful with very long lists, where scrolling without a scrollbar can be very tiresome, and...

Sven Hafner asks How to address specific PostgreSQL Schema in Glassfish ?: Hi ! I create a connection pool and connection pointing to a PostgreSQL DB. Is there a way to point to a specific schema inside the DB (like a schema parameter) ? I need to avoid a schema annotation inside the EJB code, so I can point several instances of the same application...

In the Wonderland forum, seungchan posted Module deployment and get a specific revision: Hello, I have a module that includes web pages on server admin. It was deployed fine in wonderland version on Dec.2009, but it was not deployed in current trunk version of wonderland. (rev. 4773) Since I did not get any error message when I deploy this module, I am spending hard time to solve it ...

Our current Spotlight is Janice Heiss's interview with Java Champion Adam Bien on JavaFX. Adam's opening statement: "Good UI controls and layout are the key to success. JavaFX was very strong from the beginning in effects and graphics. It was, however, initially lacking in good, "skinnable" components, but this was fixed with version 1.2. JavaFX requires writing less code while it integrates very well with existing business logic written in Java. A reason to go the JavaFX route is better maintainability, and faster development with less code..."

This week's java.net Poll asks What's the most important java.net project going forward?. Voting closes tomorrow.

We've just published a new java.net Feature Article, Maven Repository Managers for the Enterprise, by John Smart. We're also featuring Jeff Friesen's Reading Newsfeeds in JavaFX with FeedRead, in which Jeff demonstrates how to apply JavaFX's RSS and Atom newsfeed capabilities to create a snazzy little JavaFX app that can run stand-alone or in a browser.

The latest Java Mobility Podcast is Java Mobile Podcast 92: MIDP 3.0 in Depth: Tutorials and Demonstrations: Excerpts from the JavaOne 2009 MIDP 3.0 In Depth: Tutorials and Demonstrations session with Roger Riggs, Lakshmi Dontamsetti and Stan Kao.

Current and upcoming Java Events:

• February 4-6: FOSDEM 2010
• March 17-19: TheServerSide Java Symposium 2010

Registered users can submit event listings for the java.net Events Page using our events submission form. All submissions go through an editorial review before being posted to the site.

Archives and Subscriptions: This blog is delivered weekdays as the Java Today RSS feed. Also, once this page is no longer featured as the front page of java.net it will be archived along with other past issues in the java.net Archive.

-- Kevin Farnham
O'Reilly Media

editor

While benchmarking a concurrent application on Linux I ran into an odd problem worth relating. Specifically, I'm using ubuntu 9.10 with linux kernel 2.6.31-1 running on a 1x4x2 Core2 i7-920 "Nehalem" (1 package; 4 cores/package; 2 logical processors/core via hyperthreading). I'd noticed that our scaling numbers were a bit odd, with more than the usual fall off past 4 threads (it's a 1x4x2 system so we expect some fade past 4 threads, even for ideally scalable benchmarks) and more variance than I expected. The benchmark harness runs for fixed time and reports aggregate progress over the measurement interval, and of course the system is otherwise idle. As a sanity check the benchmark also reports the total user CPU time consumed over the measurement interval. Interestingly, the CPU times values were unexpectedly low, considerably less than min(#threads,#cpus) * MeasurementInterval. All the threads should stay running, or least ready, for the duration of the interval. Note too that the effect is independent of and still occurs with long intervals, so it's not a simple issue of allowing the scheduler time to steal and rebalance or otherwise converge on a state where the runnable threads are well-dispersed over the processors.

It appeared that the scheduler wasn't aggressively dispatching ready threads onto idle CPUs. Put another way there were prolonged periods where we had both idle CPUs and ready threads at the same time -- the kernel was failing to saturate the available processors.

To avoid the problem I initially tried binding threads to processors via sched_setaffinity, which provided complete relief. Still, I'm cautious about binding because it requires knowledge of the platform topology. On SPARC/CMT/Solaris, for instance, logical CPUIDs map to physical resources geographically in the following manner: the bits in the logical CPUID select, from most significant bits to least significant, chip then core then pipeline ("thread group" in Sun terminology) then strand. So if you just bind threads by "natural" order (thread N to CPUID N) then you'll end up with many threads sharing some cores and other cores completely idle, which is likely undesirable and may yield skewed scaling results. This, btw, is a common benchmarking pitfall. On Solaris/SPARC you're better off letting the kernel disperse threads onto processors as it'll balance 1st over chips, then cores, then pipelines, which is optimal for independent threads to make headway. (That policy is clearly not optimal if there's sharing -- particular if there are writers -- in which case you might win by packing the threads less "distant" from each other, for some interconnect distance metric, and assuming the increased cache pressure and replication doesn't do too much harm). Unlike SPARC/Solaris, the logical operating system-level CPUID to physical resource mappings on my ubuntu/x64 system are well-dispersed if you use natural CPU assignment, but there's no hard guarantee of that property although I vaguely recall that Intel advises a certain canonical mapping. In more detail, the logical CPUID to physical mapping on my system -- as discovered by iterating over the logical CPUID values and using the CPUID instruction to query physical resources -- is : 0 to C0S0, 1 to C1S0, 2 to C2S0, 3 to C3S0, 4 to C0S1, 5 to C1S1, 6 to C2S1, 7 to C3S1, where C is the core# and S is the relative strand# on the core.

I'm guessing that the linux kernel I'm using institutes polices that attempt to balance power with performance whereas Solaris currently optimizes for performance. After further poking through the Linux kernel sources I realized we could adjust the scheduling policy more to our liking via tunables exposed via the /proc file system. At that point I came upon the tune-sched-domains script that makes it easy to quickly adjust scheduler tunables. (Note that the script assumes bash). First, run tune-sched-domains with no arguments and examine the SD_WAKE_AFFINITY and SD_WAKE_IDLE settings. We want SD_WAKE_AFFINITY clear and SD_WAKE_IDLE set. (If I'm interpreting the comments in the kernel code correctly, WAKE_AFFINITY appears to try to place the wakee on the same CPU as the waker, presuming they communicate through memory that's already in the local cache, while WAKE_IDLE instructs the kernel to aggressively wake idle CPUs when making threads ready). If necessary, compute a new SD_ mask value and run the script again, passing the value (in decimal) as an argument to the script. These settings provided relief for the under-utilization problem.

In addition I noticed that the HotSpot JVM performed much better on multi-threaded workloads under the settings mentioned above.

While I didn't have time for the experiments, it may be the case that adjusting the LB_BIAS flag may also provide relief.

Twitterized panel says voters will put policy first

Technology or policy - which will be more influential come this year’s General Election Campaign?…

Web threats: Why conventional protection doesn't work

(author unknown)

There's a new feature called block migration available with the recently released qemu-kvm-0.12 version. Patches for this feature was first introduced back in November 2009 by ibm and allows you to do block migration during live migration. It works by copying block images to a destination asynchronously. It does this by copying the whole disk first then copying dirty blocks accumulated during the migration similarly to ram migration.

 

If you're running qemu-kvm-0.12 you can access this feature from the qemu monitor. There are actually two modes of migrating block storage.

• Live migration with complete storage copy
• Live migration with incremental storage copy . This is for the case where you're using a “linked image”. In other words, If you're using a copy on write block image based on a base image a.k.a backing file. In this case the backing file must be present on both source and destination servers. For more on backing files or base images see the following post on base images.

The commands for performing these operations are as follows. For complete storage copy, use the -b flag on the source when migrating your virtual machine as shown below.

(qemu) migrate -d -b tcp:1.2.3.4:4444

For incremental storage copy where you're using a backing file use the -i flag on the source as follows. Your backing file should obviously be in the same location on both source and destination servers.

(qemu) migrate -d -i tcp:1.2.3.4:4444

If you're in a situation where shared storage is not currently available for cost or other reasons and really would like to perform live migration, this is your best solution. Bear in mind however that your guest will be suspended during the end phase of block migration so you can expect a little downtime. I haven't personally tested this yet so give this a test run and feel free to post comments or questions.

 

Haydn Solomon06209445995077776529

There's a new feature called block migration available with the recently released qemu-kvm-0.12 version. Patches for this feature was first introduced back in November 2009 by ibm and allows you to do block migration during live migration. It works by copying block images to a destination asynchronously. It does this by copying the whole disk first then copying dirty blocks accumulated during the migration similarly to ram migration.

 

If you're running qemu-kvm-0.12 you can access this feature from the qemu monitor. There are actually two modes of migrating block storage.

• Live migration with complete storage copy
• Live migration with incremental storage copy . This is for the case where you're using a “linked image”. In other words, If you're using a copy on write block image based on a base image a.k.a backing file. In this case the backing file must be present on both source and destination servers. For more on backing files or base images see the following post on base images.

The commands for performing these operations are as follows. For complete storage copy, use the -b flag on the source when migrating your virtual machine as shown below.

(qemu) migrate -d -b tcp:1.2.3.4:4444

For incremental storage copy where you're using a backing file use the -i flag on the source as follows. Your backing file should obviously be in the same location on both source and destination servers.

(qemu) migrate -d -i tcp:1.2.3.4:4444

If you're in a situation where shared storage is not currently available for cost or other reasons and really would like to perform live migration, this is your best solution. Bear in mind however that your guest will be suspended during the end phase of block migration so you can expect a little downtime. I haven't personally tested this yet so give this a test run and feel free to post comments or questions.

 

Haydn Solomon06209445995077776529

Prepare for Guava juice

Google has published code for Java to reduce the amount of hand coding needed for commonly occurring or popular features in applications.…

The power of collaboration within unified communications

(author unknown)

Chechen terrorists did it in 2004. I said this in an interview with then TSA head Kip Hawley in 2007:

I don't want to even think about how much C4 I can strap to my legs and walk through your magnetometers.

And what sort of magical thinking is behind the rumored TSA rule about keeping passengers seated during the last hour of flight? Do we really think the terrorist won't think of blowing up their improvised explosive devices during the first hour of flight?

For years I've been saying this:

Only two things have made flying safer [since 9/11]: the reinforcement of cockpit doors, and the fact that passengers know now to resist hijackers.

This week, the second one worked over Detroit. Security succeeded.

EDITED TO ADD (12/26): Only one carry on? No electronics for the first hour of flight? I wish that, just once, some terrorist would try something that you can only foil by upgrading the passengers to first class and giving them free drinks.

schneier007450309532310700901328979773785625439308246357305965844093061575019810698619631639770871023814493407879901715447993579049502641366191865670923023975498025292815213357477338739738111202947793977007491192644654223956036108523800983464001102094138958899098089950148717746070457326502264210696210419272141155133774694659330276546615800136298213435546311595178529133238128293763777700911595207673324919906330713561583253242022650524929726241501036842225826514215307737432326876565309085800669396734532711172119777165622152303640784732684122866089000771319822753251178451622844945093205198062041625675827088190415942333517431808139590731890813604972469577963010686094777960842567741860997785813010023369110254507706749959362111844872183189428370556218384552476973301401665302629790652005984464744077624040612180889323057111807554520292441851145027272878304370626650690804022959640981200406962949592468741036736302862714568081083523107810618951706604929322182354293035776268052831064590723285507017027303404816926377971815335073966341831420727940775642769163969101207414107714991643013048373100825408853751731966837353611489403674378642905604247070415279495382968450022720703298527695103352555410993041641178448672489202395581339984336027101499814989776657917525319042203562894852963301534744399553076319313698539527814486112156201028544649015130031338131408873481814117853011607532406047252303569211003020546560359478684477416894783967708706728

One of the goals for JDK 7 is to get us to a modular platform. Getting there will be hard as it's a very interconnected code base with many undesirable dependencies between APIs and different areas of the implementation. These dependencies have built up over many years and releases. To give an example (from a couple of builds ago but mostly applicable to JDK 6 too): Suppose you are using the Logging API (meaning java.util.logging). Logging requires NIO (for file locking) and JMX (as loggers are managed). JMX requires JavaBeans, JNDI, RMI and CORBA (the JMX remote API mandates that the RMI connector support both JRMP and IIOP). JNDI requires java.applet.Applet (huh?) and JavaBeans has dependencies on AWT, Swing, and all things client. Not satisfied with this, JavaBeans has persistent delegates that create dependencies on JDBC and more. I could continue but it should be clear that this seemingly innocent use of the logging API results in transitive dependencies that envelop almost the entire platform. And just to be clear - these are just dependencies and logging shouldn't of course require any of CORBA's 1600+ classes to be actually be loaded. Think of it more like dinner for two except that she hires a fleet of buses to bring her extended family and friends to wait outside the restaurant.

The good news is that we've started to make progress over the last few builds to address many of these issues. Logging no longer requires JMX (this required an API change to be backed-out/re-visited). We're separated out the RMI-IIOP transport so you can do remote management without CORBA being present. JMX will now do its own introspection when JavaBeans is not present. JNDI no longer requires java.applet.Applet, JavaBeans no longer requires JDBC, AWT no longer requires RMI, and many more.

So where are we at? At this time we have a tentative base module that is essentially the core libraries (think lang/io/net/util/nio/security). The dependencies that used to exist from the classes in this module on JNDI, deployment code, AWT, the preferences and logging APIs, and JMX have been removed or inverted. There is a remaining dependency on XML parsing (from java.util.Properties) and that will be solved in time.

All things Swing, AWT, 2D, etc. are grouped into a tentative client module. The APIs in this module are deeply interconnected and so pose a big challenge. There are still a few dependencies from other modules (like web services) on client that will require work but ultimately it should be possible to chop off the head, say when deploying on an embedded device.

We have a several fine-grain modules that could potentially be grouped, maybe into coarser grain profiles in the future. Logging, RMI, JSSE (SSL), SASL, JDBC, JNDI, LDAP and other JNDI providers, PKCS11 and other security providers to name some of them. JSSE is a good example of work done to decouple it from other areas of the platform. One would think it should be possible to do secure networking without requiring the world but, as SSL can negotiate to use Kerberos based authentication, it was tied to Kerberos/JGSS. For this case, the dependency is now optional. If Kerberos is installed then SSL will include the Kerberos cipher suites when negotiating the security context. When not installed it won't negotiate to use Kerberos.

I mentioned CORBA above as it is often used as the whipping boy by those that are critical of the compatibility baggage that the JDK carries. A potential module that has been suggested is a compatibility module for deprecated, legacy, unloved and other baggage. Good work from Sean Mullan and Vincent Ryan in jdk7 b78 removes the dependencies on the deprecated security classes so that they don't need to be in the base module. Other potential candidates are the legacy sun.io converters. We should have ditched these years ago but there are still JDBC drivers that haven't removed their dependencies. We've also got many classes in sun.misc that aren't used anymore but we can't remove them because there may be naughty applications out there using them directly. Legacy protocol handlers and content handlers are other candidates. I'm sure the reader can think of others.

In the above it is worth pointing out that modules aren't necessarily aligned on package boundaries. It's clearly desirable for a module to contain all the classes that are in one or more complete packages but there are many cases where this isn't not possible. I mentioned JavaBeans above and that is a clear case where one has to separate out the property event support and annotations from the introspection and other classes that tie one to the client area. APIs such New I/O and Logging have management interfaces and it makes a lot more sense to group the management interfaces into the management module along with with JMX and java.lang.management. I mentioned the separation of the IIOP transport from the RMI Connector above. For that one, the rmic compiler will generate the stubs and ties to the javax.management.remote.rmi package but we wouldn't want to group these into the management module as it would create a dependency on CORBA. If one were to split up the base module further then it would mean looking at packages such as java.util that contains a lot more than the collection APIs.

I hope the above gives you a feel for the work that has been happening in jdk7. A more modular JDK should get us closer to our goals to improve performance (download and startup time), enable the platform to scale-down, and more. For those interested in diving into this further then the Jigsaw project page and mailing list are a good starting point. The jigsaw/tools repository has the ClassAnalyzer tool that we've been using to analyze dependencies and guide the changes. The tool consumes module definitions and generates several files per module, including the list of classes, and dependencies. There are summary files in various forms, including DOT files for those interested in visualizing the dependencies. Much of the work mentioned above can be thought of as removing edges from the dependency graph. Mandy has been working on the next step, the build changes so that the JDK build will generate modules rather than rt.jar. This will take a few steps to get there. Initially the build will generate JAR files but ultimately we will of course transition to a better container format.

alanb09672041804595895303

That is all.(author unknown)