Microsoft Live Labs

With the popularity of the recent collections from the space shuttle Endeavor, we continued to receive feedback that a few people with specific configurations were unable to install the Photosynth viewer.  We’ve made a few tweaks to the installer that should resolve the majority of those issues, including support for 64-bit systems.  If you were one of the unlucky few who couldn’t try Photosynth before, give the new installer try.

There’s no new functionality, just a better install experience, so you don’t need to do anything if you already have Photosynth installed.

As the seven-member crew of the Space Shuttle Endeavour prepare for their return to Kennedy Space Center tomorrow morning. We are pleased to announce an update to our collaboration with NASA.

Prior to docking at the international space station the Space Shuttle Endeavour did a complete somersault enabling astronauts in the International Space Station to photograph the shuttle"s bottom-side. The photos were then sent back to NASA for analysis.  The Space Administration has been kind enough to share those images with Microsoft Live Labs. We have taken the photos and created a “synth” so people around the world can take part in the NASA experience.

So here it is, your chance to take first-hand look at what you might see on a space-walk .

Click here to install the Photosynth viewer and experience for yourself.

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

Today, we’re announcing an exciting collaboration with NASA using Photosynth technology, to create three-dimensional environments of preparations for the launch of space shuttle Endeavour.  For the first time ever, viewers can experience unprecedented views of shuttle Endeavour on the launch pad, interior and surrounding area of the Vehicle Assembly Building, and the return of previous flight Shuttle Atlantis atop a 747. Photosynth is able to capture the depth and complexity of the shuttle’s structure, providing exclusive images to viewers around the world enabling the unique ability to explore this NASA mission.

Click here to install the Photosynth viewer or visit our video section to see the technology in action.

In addition, Live Labs has collaborated with MSNBC to create a special multimedia tour of the Kennedy Space Center at http://spaceworld.msnbc.com

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

Over the last 6 months we’ve been working with the BBC on a time limited technical trial of the Photosynth technology in support of the "How We Built Britain" television series. Viewers will be able to explore synths of Ely Cathedral, Burghley House, the Royal Crescent, Bath, the Scottish Parliament Buildings, and Blackpool Tower Ballroom at http://labs.live.com/photosynth/bbc/. The BBC will also have units on location at each of the historic sites to collect images from tourists visiting the sites that we will add to the synths in the weeks to come. Be sure to check out Trafalgar Square that also features some fantastic historical photography to contrast how people interacted with the locations in the past and present and check back every few weeks to see how they evolve.

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

In March of this year Microsoft Live Labs Architect, Blaise Aguera y Arcas spoke at the TED (Technology, Entertainment, Design) Conference in Monterey, CA. Blaise demonstrated Seadragon and Photosynth to an enthusiastic audience and painted a vision for how these technologies will evolve in the months and years ahead. Watch the short video above to see some of the amazing ways these technologies will change the way you interact with your images in the future, and visit the Photosynth site to try the latest collections from the team.

Over the last few months we’ve been gathering feedback both internally and from our partners on the authoring experience for Photosynth. The experiments have yielded some fantastic results that we anticipate sharing on the site in the weeks and months ahead. Today we’d like to share one such example from our friends at MSN Korea. One brisk Seoul winter month, over 4,000 photos were submitted by local Koreans to help construct this synth of Gyeongbok Palace, one of South Korea"s finest national treasures. Originally built in 1394, it is the largest palace of the Joseon Dynasty. At its height in the mid 1800"s, it covered 330 buildings and over 4 million square feet as the grand home for the royal family. This is the first synth we’ve shown that contains a mix of professional photography and candid photos from other people using their own cameras.

Enjoy exploring the Palace and check back soon for even more Photosynth goodness.

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

It"s been great for us to watch your comments, suggestion and ideas around Photosynth. As you know, this is a long term project and the team continues to make progress on a daily basis to add more functionality to the underlying technology. After we released the technology preview in November we saw a number of requests for a version that would work in Firefox. Live Labs is committed to making our technologies available to the widest possible audience, and today we"re happy to announce the availability of the Photosynth Firefox plugin. We know that you are waiting for the day when you will be able to create your own collections, and that will happen, but in the meantime look for more beautiful new examples of Photosynth in action over the coming months.

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

It"s been great for us to watch your comments, suggestion and ideas around Photosynth. As you know, this is a long term project and the team continues to make progress on a daily basis to add more functionality to the underlying technology. After we released the technology preview in November we saw a number of requests for a version that would work in Firefox. Live Labs is committed to making our technologies available to the widest possible audience, and today we"re happy to announce the availability of the Photosynth Firefox plugin. We know that you are waiting for the day when you will be able to create your own collections, and that will happen, but in the meantime look for more beautiful new examples of Photosynth in action over the coming months.

Adam Sheppard
Group Product Manager

Live Labs | Microsoft

Free motion is another somewhat hidden feature of Photosynth. We’ve documented the keys in our help section, but we haven’t gone out of our way to advertise it. If you’ve ever played Doom or a similar pc game, we use a very similar (but not exact and I’ll get into why down below) key layout.

With your left hand, use the keys w,a,s,d to move forward, left, back and right. And, Since we can go up and down unlike in Doom, we use the e and c keys, respectively, for this motion.

With your right hand use the p.l.;.’ keys to rotate up, left, down and right.

When used in combination, these keys let you move in any direction through the scene. One of the best ways to start is to hit ‘e’ once to pop up above the scene a bit, then use ; to rotate the scene down so you can see it. Then you can use your left hand to roam around above it.  If you get lost, hit ‘enter’ to center back on the currently select photo or hit 0 to go to the home photo.

This is especially effective in the larger sites of San Marco and St Peters. It lets you get more of a feel for the 3D nature of the scene and to take in what we feel is the beauty of the point clouds.

It’s also shows interesting potential in the Faigin Gallery. Click here to see where I’ve navigated overhead to see the footprint of Gary’s studio.

If you think this is cool let us know. We left the feature in the app, but didn’t expose it through the UI, because it’s just not that compelling. Don’t get me wrong, the motion itself works fine – all of the movements are consistent and move you through the space as you would expect. The problem comes into play with the data and the way we present photos. Since we don’t have any constraints in the current model, when you’re moving around freely, it’s very easy to go through photos and get lost. In fact, when we first designed the UI, we used the arrows for rotation, but in testing we found that it confused people and didn’t add to the experience. That was why we switched the arrow keys to go to a next photo in a given direction. Eventually we’d love it if you could move around freely and have photos automatically display that represent where you are looking.

Jonathan Dughi

Photosynth Program Manager 

Live Labs | Microsoft

If you’re as mesmerized by exploring some of the Photosynth collections as we are, you might be interested to know that there’s a hidden autoplay function in Photosynth.  Launch any collection, then press the “1” key. Every 7 seconds Photosynth will advance to the next image in a tour that will take you through every photo in the set. Clicking anywhere interrupts the tour, but you can use your mouse wheel to zoom while the tour is going on if you like. You can almost run it as a screensaver... just maximize IE using F11 and minimize the sidebar tab and the application will fill your screen.

There’s even a back door to control the transition time. If you add an “&autoPlayInterval=5000”  argument to the end of your URL, you get a slide show that transitions every 5000 milliseconds, or 5 seconds. (The minimal legal value for this parameter is 3000). Here’s a sample URL: http://labs.live.com/photosynth/view.html?collection=FaiginGallery/index1.sxs&autoPlayInterval=5000

Be careful – watching Photosynth do its thing can seriously impact your productivity.

Cheers,

David Gedye
Seadragon and Photosynth Group Manager

Live Labs | Microsoft 

It’s been just a few short months since we announced Photosynth at SIGGRAPH 2006 in Boston and boy have we been busy!

Today we’re very happy to announce the first public release of our Photosynth Technology Preview. For those of you who’ve provided encouragement, ideas and spread the word we thank you. We hope that you’ll enjoy our current work and continue to help us shape the technology as we push onwards. For those of you who found yourselves here after your buddy sent you a link to our website saying, ‘You’ve got to check this out!’, read a little about what we’re doing, try it out for yourself, and obviously tell everyone else you know. :)

We thought you might have some questions about this release and we’ll do our best to answer them. If there’s anything we haven’t addressed feel free to post to the comments in the blog and we’ll get to them as quickly as we can.

• Q. What does Photosynth do?
• A. Photosynth combines hundreds or thousands of regular digital photos of a scene to present a detailed 3D model, giving viewers the sensation of smoothly gliding around the scene from every angle. The scene can be constructed regardless of whether the photos are from a single or multiple sources. It’s like a hybrid of a slide show and a gaming experience that lets the viewer zoom in to see greater detail or zoom out for a more expansive view. By viewing the photos in a 3D context you are able to get a better sense for the place where they were captured.

• Q. What exactly is a ‘Technology Preview’?
• A. In essence a Technology Preview is a sneak peek at good things to come. Microsoft Live Labs is very much focused on pushing the state of the art of the internet forward and showing what’s possible today. We like to do things quickly and in a very collaborative way. Over time you’ll start to see more and more of our technologies work their way into great products across Microsoft, but in the meantime we want to get your feedback on what we’re doing right, and areas where we can improve. Being on the cutting edge unfortunately comes at a little bit of a cost so if your computer is more than a couple of years old, there"s a chance that your graphics drivers may prevent Photosynth from running properly. If weird stuff happens, please install the latest drivers for your graphics card and try again.

• Q.When can I build my own collections?
• A. We’re working on it! We want to provide this capability as soon as we can but there are some real technical hurdles to solve before we’re ready for primetime. Today each and every image in a collection needs to be compared against each and every other image in order to see if it’s a match for the scene undergoing reconstruction. Processing to build a collection can take hours or days in some cases. We have lots of ideas on how to improve this processing time but we need some time test some of these ideas and make the tools easy for people to use. In other words, stay tuned and enjoy the collections we’ll be releasing in the months to come until we’re ready for our next big update.

Thanks and enjoy!
The Photosynth Team.

Ever since Photosynth was announced at SIGGRAPH, people have been asking about the relationship to the Photo Tourism paper. I thought I would write up a bit of history surrounding this multi-team project.

About six months ago, a Microsoft researcher, a PhD student, and a University professor invented a new photo browsing technology called “Photo Tourism”. The technology was so cool that anyone who saw it wanted to be part of the project. Noah Snavely, a student at the University of Washington (UW), had single-handedly coded a rich prototype under the joint supervision of Steve Seitz (also from UW) and Rick Szeliski from Microsoft Research (MSR). The question was: what were we (Microsoft) going to do with such a rich concept? We had joint IP between a non-profit and a for-profit organization for what could be a very hot product. As a Research Area Manager at MSR, my role was to promote both Microsoft’s business interests and the open MSR-academic collaboration tradition that had made this project possible in the first place. Unfortunately, no product team could immediately absorb this technology. The solution came from Live Labs, a new research group with a product focus.

Live Labs is both a research and a rapid development organization, with a commitment to produce high-impact prototypes related to the Internet. This emphasis distinguishes it from the more general and academic Microsoft Research. It also provides good complementarities between Live Labs and universities. Between January and March, Live Labs had two growth spurts. Late in January, Live Labs completed the acquisition of Seadragon, a Seattle company of 10 employees headed by Blaise Aguera y Arcas. Later in March, Gary Flake convinced me to head the research arm of Live Labs. I transferred from MSR along with a dozen other scientists and engineers. Both events would influence the future of Photo Tourism by providing the means and motivation to take it to the next step.

Seadragon had also developed a photo browsing technology. It was very different from Photo Tourism. Seadragon provided seamless zooming around collections of pictures and efficient multi-resolution streaming transport between clients and servers. Photo Tourism matched pictures, found logical transitions between them, and built a 3 dimensional interraction model. Seadragon and Photo Tourism were a marriage of technologies made in heaven. With rapid prototyping in mind, we entered a mutually beneficial partnership between the University of Washington, Live Labs, and MSR. The symbiosis of Photo Tourism and Seadragon became the Live Labs project called Photosynth, a marriage of technologies as well as cultures. The PhotoSynth team is made of Seadragon, the original inventors of Photo Tourism, and several researchers from MSR and Live Labs. Working together they built a prototype which enables users to remotely navigate and explore a 3D model, which is automatically built from a collection of pictures. The prototype was previewed at SIGGRAPH and will be released on the Web in a few weeks.

Patrice Simard
Chief Scientist
Microsoft Live Labs

Hi folks, it’s been a few weeks since we announced Photosynth at SIGGRAPH 2006 and we thought it might be useful to give you a quick update on what we’re up to.

Firstly, we’ve been overwhelmed with the positive responses in the blogosphere and mainstream press, with coverage including the BBC and CNET as well as Digg, Slashdot and Motley Fool. Live Labs is a small, new team at Microsoft and it’s gratifying to see that the online community shares our vision for the potential of Photosynth. One of the things we felt got a little lost in all the news was that Photosynth was born out of a deep collaboration between Rick Szeliski at Microsoft Research, Noah Snavely and Steve Seitz of the University of Washington. Noah and Steve continue to be deeply involved in helping us evolve Photosynth and exemplify the kind of cross collaboration with academia that we aspire to continue at Live Labs. You can read a lot more about their work at the Photo Tourism site.

Siggraph was a blast! We had the chance to meet hundreds of graphics professionals and enthusiasts and hear their reactions to Photosynth first hand. Rick, Noah and Steve’s presentation of "Photo tourism: Exploring photo collections in 3D" was extremely well received and we’ve been inundated with requests from people with various different applications of the Photosynth Technology.

One of the questions we’re often asked is when people will be able to create their own 3D collections. This is something we’re absolutely committed to but will not be available in our first external release. Photosynth currently requires a large number of CPU cycles to perform the matching between images and for larger datasets this can take hours or sometimes days of processing. We are exploring a number of methods for drastically reducing this processing time but want to ensure that people realize that we’re a technology that is evolving in real-time and we want to incrementally build towards the vision of a world of interconnected images through collaboration and participation from the online community.

To that end we will be releasing an ActiveX browser control this fall that will allow you to explore a number of interesting, processed collections. We showed an early build of this at our SIGGRAPH booth and many people gave us feedback on the interface that we’re currently incorporating. This build will also include the amazing multi-resolution, zoom and navigation capabilities of SeaDragon which is an experience unto itself!

So that’s all for now, stay tuned for updates from myself and some of the other folks on the team in the weeks to come and thanks again for stopping by!

Adam Sheppard
Group Manager
Microsoft Live Labs.

Welcome to ‘Photosynth’, Microsoft Live Labs first major Technology Preview. The team is very happy to give you a sneak peek at our latest work which we’ll be demonstrating next Wednesday 8/2 at SIGGRAPH 2006 in Boston.

Photosynth takes a large collection of photos of a place or object, analyzes them for similarities, and displays them in a reconstructed 3-Dimensional space. With Photosynth you can:

• walk or fly through a scene to see photos from any angle;
• seamlessly zoom in or out of a photograph even if it"s gigapixels in size;
• see where pictures were taken in relation to one another;
• find similar photos to the one you’re currently viewing;
• explore a custom tour or see where you’ve been; or
• send a collection to a friend.

We plan on releasing some collections for you to see and experience yourself in the very near future, so add us to your RSS reader and check back often for updates. We’d love to hear your ideas and suggestions and help us shape Photosynth over the months and years to come. In the meantime, take a look at the videos and learn a little bit more about how Photosynth works.

Curious about who we are? We’re a team of Microsoft scientists and engineers on a mission to fundamentally rethink the internet experience, experiment with entirely new paradigms and change the world through ideas and software. Our founder Dr. Gary Flake, started Live Labs in February 2006 and since then many members of the team have been working around the clock to develop their concepts into a reality. Photosynth is just the start of what we hope will be a series of technologies that will change the way you use the internet.

Thanks for visiting!
The Photosynth Team.

Earlier this week some of us met with Channel 9 and its sister Channel 10 to talk about the technology, the team and life on the Live Labs team.

See Live Labs Architect, Blaise Agüera y Arcas talk through the technology behind Photosynth at the new Live Labs Offices at the Smith Tower in Seattle.

Or listen to Group Manager, Adam Sheppard talk with Laura Foy about how you might use Photosynth in the future, the Seadragon technology that powers it, and the cross group collaboration that made it all happen in a matter of months.