Project Looking Glass

Print-friendly Version

October 26, 2004
Guests: Hideya Kawahara, Paul Byrne, and Deron Johnson
Moderator: Edward Ort (MDR-EdO)

This is a moderated forum.

MDR-EdO: Welcome to today's Java Live chat on Project Looking Glass. Project Looking Glass is an exciting exploration into the next generation desktop. It's a 3-dimensional desktop with clever features such as being able to flip over a window on the desktop and paste notes on the back. Project Looking Glass has been released to the open source community to benefit from community-wide ideas about the functions and uses of such a desktop. In today's chat, you'll have a chance to ask questions about Project Looking Glass and get answers from the members of the core team: Sun engineers, Hideya Kawahara, Paul Byrne, and Deron Johnson. Our guests are ready, so let's begin -- who has the first question?

Haroldo Lage: How it is intended to interact with the program? Is the mouse is enough?

Paul Byrne: By default users can interact with Looking Glass using a mouse. We will also work on keyboard shortcuts for advanced users and 508 compliance.

ttilt: Where's the project currently stored and what kind of versioning control is being used?

Paul Byrne: java.net, all the source is under CVS control.

Eric: When will the technology be publicly available?

Paul Byrne: A developer release has been available since June this year. All source is available. We don't have any firm time scales for release of a full product, although we have committed to having elements in JDS 5.

Handley: What are your current plans to integrate with JDS? Do you have a time frame? Or are there other partners that will take this to market?

Hideya Kawahara: We are still in an early planning stage, but we hope JDS 5 will include some elements of Project Looking Glass.

fooman: What are some of the things that could be done in a 3D desktop (for example, flipping over a window) that can't be done in a 2D desktop?

Paul Byrne: The goal of the project at the moment is allow everyone in the community to experiment with and add 3D features to determine what works well in a desktop and what does not. Currently we can scale windows to any size without the application re-rendering and we can push windows into a bookshelf view in which you can see the application's contents but it takes much less screen space. We also have multiple desktops which we can order in various ways and transition between smoothly.

Haroldo Lage: Does Looking Glass use any specific hardware acceleration?

Paul Byrne: Looking Glass uses Java 3D to get hardware acceleration from the graphics hardware in the machine. Java 3D (in most cases) uses OpenGL at the lowest level.

MX5: Since it's a desktop, does it override any other desktop from the OS or do they work together?

Deron Johnson: LG has two modes. The first mode is a development mode. It is run via the script lg3d-dev. This mode is independent from the underlying OS. It runs on any system which supports JDK 1.5: Linux, Solaris, Mac, and Windows. The primary of purpose of this mode is to support the development of 3D applications. This mode does not support windows of the underlying OS window system (such as X11 windows on Linux). The development mode is started from within the OS's base window system. For example, on Linux you can run lg3d-dev from within GNOME. The other mode is the "session" mode. This is started by exiting the base window system and running lg3d-session. This mode provides the ability to start windows of the base window system. For example, on Linux you can run X11 applications in this mode. Please note that the session mode is still under development and is somewhat buggy, so it is not quite as far along as the dev mode.

Eric: Are there any plans for integration with future Microsoft OS's?

Hideya Kawahara: No.

Gugo: When will the first prototype/beta version/version be available for download on the web?

Paul Byrne: It's already there, check java.net.

Gugo: It's obviously Java, but is there a preferred hardware for the technology or what is the performance hit for using the technology?

Hideya Kawahara: It requires good OpenGL support, which is used by Java 3D. On the Linux platform, it seems that NVIDIA's graphics card and driver works better.

dwalsh: What platforms do you believe Looking Glass will support (Linux, Windows, Solaris)?

Deron Johnson: Our current plans for the near term are to support Linux and Solaris x86. Solaris SPARC support is possible, but we'll need some help more help to make that happen. Windows integration would require some way of getting at the bits of a Windows window. My understanding is that Microsoft used to support this but this feature was removed in XP. If anybody knows how to do this in Windows I'd be eager to hear about it. Mac support is theoretically possible because their system is X based, but it would take someone more familiar with Macs than I to look into the details.

Guest: Are there plans to use virtual reality?

Paul Byrne: You can actually think of the current project as 2 things. 1) A platform for exploring potential 3D desktop features, 2) A sample 3D desktop. The platform itself does have support for virtual reality systems such as caves and power walls and can support 3D input devices (there are some holes in the implementation which need to be fixed). Our sample desktop does not use virtual reality though because we are trying to keep the desktop close to existing desktops to make user transitions easier. So in summary the platform will support VR but in the short term our desktop won't take advantage of it.

ttilt: Any estimates on when an initial test version would be out?

Deron Johnson: The test version is already out. Go to java.net for details.

Guest: I've seen the demo; it's a very interesting product. I'm currently working on a 3d mouse. It is intended to in the future use some sort of device of that kind?

Deron Johnson: Yes, we want to support 6 degree of freedom input devices in the future, but it isn't in our near term plans. But it's on our long term road map.

GlassEye: Can you share with us some of the more interesting ideas that have come up as a result of open sourcing the developer release?

Hideya Kawahara: Yes, lots of interesting discussions have been going on using the forum and the Wiki pages. Please check them out. The intent of open sourcing the technology is to develop next generation desktop together. If you have any cool idea, please share it with us. We'll appreciate it!

3Denthusiast: How many developers are currently working on this project and do you have a need for any other particular specialties (Macs for example)?

Paul Byrne: There are currently 4 developers, 3 at Sun and 1 community developer submitting code to the core. There are also a large number of community members submitting code to the incubator projects and suggesting ideas. The community members on the forums are also very active in supporting new users. We would love to see more developers working on the project, either on core features, new desktop ideas or even 3D applications user interfaces.

Guest: Will this project include multiple desktops? As well as being able to drop objects into the background of 3d space?

Deron Johnson: What do you mean by multiple desktops? If you mean multiple screens, we do plan to support multiple screens. The initial version of this will support multiple "separate" screens in which you can move the cursor between screens but you cannot move windows between screens. Eventually we plan to support multiple "combined" screens. In this mode all of the screens behave as one giant virtual screen and you can move windows back and forth between screens. This is the same as the Xinerama mode of X11 or NVIDIA TwinView.

groenveld: Since Java3D and OpenGL already exist for Solaris SPARC, what more help is needed to support LookingGlass on the Blade 1500/2500 workstations?

Paul Byrne: Looking Glass uses the Xorg Xserver to support X applications within the 3D environment. The Xorg server is not currently ported to Solaris SPARC. However, the developer mode of Looking Glass will run on Solaris SPARC, but you won't be able to run X applications in the environment.

sunny: I heard Microsoft is planning to include 3D features in next release of Windows. So how does will the Glass project overcome the competition?

Hideya Kawahara: We are betting on the open source development. By join-forcing our creativity, I believe we can develop extremely cool stuff. This is our huge advantage.

Gugo: Talking about the test version, are you experiencing any delays in the events like mouse clicks, etc.? Still main concern being performance.

Paul Byrne: We are mindful of performance issues and are being careful not to be process hogs. Community members have reported running Looking Glass successfully on 750Mhz PIII with a GeForce2. We have not done any detailed performance analysis yet so there is almost certainly room for improvement.

nomad: What GNOME applications run clean?

Deron Johnson: Right now, only fairly simple X applications run clean such as xterm, freecell, and gnome-calculator. Emacs seems to run okay, with a few bugs. Mozilla sort of works, but it definitely has usability problems. If you send a request to deron.johnson@sun.com I can send you the latest testing status for our top applications. Our top applications include Nautilus, Mozilla, StarOffice, Realplay and NetBeans.

Gugo: Can you share your project plan in a little more detail? What are the major milestones that the project is planning to hit?

Paul Byrne: We plan to issue 'stable' releases every 3 months or so. The next one should be in late December and the goal is to have applications such as Mozilla, Netbeans, Evolution and OpenOffice fully functional. I've also promised to post the detailed schedule to the project website once it's finalized, should be up in a week or two.

Haroldo Lage: Is there a possibility of helping in the project? Beyond testing, using and stimulating its use?

Deron Johnson: There are numerous ways to get involved. We'd like people to let their imaginations run wild in inventing and experimenting with new types of 3D applications. We need help debugging and helping to get more X11 applications to run clean, we need people to experiment with 3D user interface techniques. Perhaps the easiest way to get involved is to beat the hell out of the system and report bugs.

Guest: How far along is the documentation for future developers to get underway? Do you guys need heavy contribution for documentation creation or do we already have a broad knowledge base available?

Paul Byrne: We have a couple of documents that talk about the architecture on the project website and we are focused on getting the javadoc up to date and complete. There is certainly much more that could be done so any contributions you want to make would be superb.

GlassEye: How do I get code into the Looking Glass incubator for sharing with other developers?

Hideya Kawahara: First, please send a message to the forum describing your idea, so that you can find people who may be interested in working with you. Once we receive the necessary paperwork, we will give you a Developer role. Then you'll be able to put back your code to the incubator area.

Eric: Have you done any testing on the Sun Workstations? With what results?

Deron Johnson: We've only tested LG on Linux PCs so far. But we have near term plans to get it up and running on Solaris x86. We anticipate this effort to take only a couple of weeks. Supporting Solaris SPARC machines is a much more involved task. The reason is because the Solaris x86 X server is extremely similar to the Linux one. However, the Solaris SPARC X server is from a completely different code base. We currently estimate that it will take someone about 6 engineer months to do a port to Solaris SPARC.

neomac: Will it support Mac OS X?

Paul Byrne: Project Looking Glass requires JDK 1.5, Java3D 1.3.2, JAI and the Xorg server today. Once those underlying pieces are available on Mac, Looking Glass should run. Integrating cleanly with the Aqua would require some changes in the X integration code.

Guest: Hi to all. What is the supported OS for Looking Glass?

Deron Johnson: In the "session" mode we currently only support Linux. The "dev" mode runs on any O.S. that supports JDK 1.5.

Gugo: What is the biggest obstacle for the project now? Where does it need the most help from open source community?

Deron Johnson: Our biggest obstacle right now is our biggest opportunity: we need more community involvement. We need developers working in all areas, such as 3D application development and X application debugging.

patrickmck: Earlier, someone said it was possible to transition between multiple desktops; I wondered if the different ones were functionally different; that is, did different tasks better or if it was merely a matter of aesthetics.

Paul Byrne: At the moment the different desktops are just aesthetic. Although they have different shapes, for example we have 1x4 desktops which are flat, we have a 4 screen circular desktop and we even have a desktop with 3D geometry as the background (penguin on icebergs). The backgrounds can certainly have some functional value as well, I'd be very keen on someone exploring this further.

Haroldo Lage: In the demo I've seen a CD player with extensible 3D use, do you plan to launch together a group of applications that use 3D facilities?

Hideya Kawahara: We'd like to work with the community to develop more applications. There are a few applications already in development in the incubator area. It would be great if you could join us!

Guest: Have ether of the KDE or GNOME groups shown interest in incorporating this into their offerings?

Deron Johnson: We've had some discussions with people in these groups and they seemed very interested in it. But my understanding is that their current focus is getting 2D transparency to work in these desktops. We are hoping that after they achieve this they will have more time to spend on 3D issues.

ttilt: Since the session version replaces the window manager, does it support other features, at this point, that a window manager would normally support like a task bar?

Paul Byrne: In Looking Glass we call the Window manager the Scene Manager, after all it's managing 3D applications not just windows ;-) The sample 'Glassy Scene Manager' on which we are working does have a taskbar and a community member has contributed a start menu although it's not integrated into the demo yet. The Scene Manager is not yet ready to replace an exist window manager, we are still exploring various ideas.

sunny: Will Looking Glass be either part of Solaris OS as desktop, similar to Gnome/CDE, or bundled with JDS?

Deron Johnson: We are currently planning to put elements of LG into JDS 5. Elements will eventually make their way into Solaris x86 as well.

gheagerty: Hi. A couple of questions: (1) How is/will LG be licensed? GPL? (2) Very interested in contributing to the SPARC port - who should I contact?

Paul Byrne: The bulk of the code is GPL, the Xorg changes are MIT. Any help you can provide with the SPARC port would be great, contact Deron Johnson for more information.

3Denthusiast: Within the current OS strategy, do you see this moving into the mobile arena?

Deron Johnson: We have purposefully limited our 3D client API to a subset of graphics features which could be implemented on OpenGL ES on micro devices. Rather than supporting the full gamut of Java3D features, which allow the construction of extremely complex virtual reality spaces, we are being fairly conservative with what we plan to support with the intent that this will make it easier to port LG 3D applications to devices such as PDAs and cell phones. However, it should be noted that the GUI design issues for implementing a 3D application on a cellphone are substantially different from implementing one for a desktop. But it is our hope that we can migrate significant elements of the LG platform into the microdevice space in the future. But we are not actively working in this area now. It would be fantastic if someone from the community could get involved to help with this.

Gugo: Talking about 3D applications, is there a way to 'port' current applications to 3D, or the only way to create 3D applications is to start from scratch?

Paul Byrne: For a full 3D application interface you would probably need to rewrite the applications interface. For Java Swing applications we may be able to use a 3D look and feel to give the illusion of a 3D application but there will some caveats.

Handley: Where did the original Looking Glass idea come from? And do you see a real need in the market for it?

Hideya Kawahara: The original idea of this two-and-a-half dimension desktop came from a hobby program I was playing with, which displayed X windows in a 3D space. I believe the key is this 2+1/2 approach, which doesn't impose a hard-to-use experience like virtual reality based 3D desktop environments. As long as we keep this in mind, I believe the project provides a nice way to leverage the current hardware capability and there is a good market for it.

fcassia: PLG currently lacks "mindshare". Have you guys thought about perhaps integrating with other more established projects and "piggybacking" on its current mindshare? (Calling Looking Glass "Gnome 3D", for instance, would be nice and would give it instant recognition).

Deron Johnson: It depends on how you define "mindshare." So far, the level of interest from the community has been extremely high. For example, LG is currently the number one java.net project in terms of both membership and discussion activity. We are very interested in getting more involved with the GNOME and KDE communities.

GlassEye: I read in a recent article on java.sun.com that you are exploring the possibility of providing a 3D look and feel for Swing. But that's not really making Swing components 3D-aware, right? When will we have a real 3D Swing set of UI components?

Paul Byrne: You are correct the 3D Swing look and feel would not make applications a first class 3D application and there are a number of limitations, such as the inability to render the Graphics2D content. We are talking to the Swing team about a full feature 3D Swing but there are no plans yet.

Haroldo Lage: Increasing the desktop idea. Would with the current LG plan be able to implement a common desktop area between different computers?

Paul Byrne: Using Looking Glass as a collaborative tool between multiple machines is something I'm very interested in and I think is very achievable. The code already includes a mechanism for sharing the 3D content across a network, we need to add the ability to broadcast the content to multiple machines and to have multiple user input.

nomad: Do Java applications have less usability problems than native GNOME applications?

Hideya Kawahara: I think that UI design matters more than what language we use. Talking about 3D applications, I think that we need aggressive investigation. Java's productivity gain will be very helpful with that regard.

fooman: Perhaps even a spherical desktop?

Deron Johnson: The extremely cool part of what Hideya has done is to give us an extra dimension to our desktop creativity. LG gives us a vehicle for exploring ideas such as a spherical desktop. It's like a brainstorming environment. Most of the 3D ideas people will come up with won't work, but there will be a few that will be pure gems.

patrickmck: Burke's knowledge web has a spherical desktop, in fact a lot of them ;)

Deron Johnson: I'd like to find out more about this. Please send me a reference to it.

fcassia: When you refer to JDS 5 do you mean JDS Linux R5? What is the expected timeframe for this? (I'm currently running JDS R2.)

Paul Byrne: Yes JDS Linux R5. The plans for this are not yet finalized.

gmgilmore: Do you have any goals or a hopeful timeline for some of the 3D and X application development to be done?

Deron Johnson: Paul has promised to send out a copy of our current schedule.

fcassia: Sun has a great product in the Java Instant Messenger client, already 100% Pure Java. Any chance of getting that open source (at least the client) and made part of the PLG environment?

Hideya Kawahara: That is an interesting idea. I'll ping them to see any possibility. If you find any other interesting projects that we may be able to leverage (especially open source projects) please let us know.

Guest: Has there been any consultation with 3D content creation companies as to the "look" of LG?

Paul Byrne: We have not talked directly with any 3D content companies to date. However we do have an HCI team doing user studies to determine how users interact with the 3D environment. We are also looking at building a toolpath to allow content developers in 3D tools such as Blender, 3D studio max or Maya to build content directly for Looking Glass. 3D content from these tools can already be loaded into Looking Glass, the next step is to allow the artist to define the animations and user interaction events.

gmgilmore: Can these features (sending to bookshelf) be implemented in any way other than direct user involvement with the mouse?

Hideya Kawahara: Yes, it is completely programmable.

Guest: Can LG interact with a database?

Paul Byrne: You could certainly build an database application with a 3D interface.

EdO: Well, we've quickly come to the end of our session. I'd like to thank everyone who participated today. I thought we had a nice range of questions. And of course, I'd especially like to thank our guests Hideya, Paul, and Deron for their answers.

fcassia: Thanks guys :)

MDR-EdO:: Thanks to everyone.

Deron Johnson: Thanks for attending. Be sure to go to java.net to find out more about LG. And if there are any questions we haven't answered please join us on the LG discussion forum.

seedling: Thanks all.

Haroldo Lage: Thanks for the opportunity of chatting with the developers :-D

Gugo: Thanks guys, can't wait to test it out.

Paul Byrne: Thanks for all great questions, if we did not get to your question or if you think of something else to ask please post to the Looking Glass forum. If you want to contribute to the project, either with ideas, code, or applications, we would love to have you.

Hideya Kawahara: Thank you to all of you! As Paul said, please go visit the forum. We're looking forward to seeing you there again!

MDR-EdO: Last moderator (me) signing off. The forum is now unmoderated.