Tag Archives: augmented reality

Bringing Machine Vision to Olympic Judging

If you’re like me, your favorite part of the Olympics is watching athletes from all over the world come together and compete to see who is the best. For many situations it is easy to clearly determine who is the best. The team that scores the most goals wins at Football (a.k.a. Soccer). The person who crosses the finish line first wins the 100-meter Dash. The swimmer who touches the wall first wins the 200-meter Breaststroke.

Victims of Human Error (and Bias)

However, is some cases, determining what happened is less clear. All of these cases involve subjective human judgment. I am not just talking about judgment regarding stylistic components; I am talking about judgment on absolute principles of scoring and penalties. As a result, athletes (who have trained for tens of thousands of hours over years of their lives) are often at the mercy of human judgment of motions that are almost to fast to observe. A few examples:

  1. A sprinter can be disqualified if she or he kicks off the starting blocks before the sound of the starting gun could potentially reach him or her
  2. A boxer may miss a point because he punches and connects too quickly
  3. A diver or gymnast can receive unwarranted penalties (or conversely, not receive warranted ones) because human judges misperceive the smallest of angles during an movement that takes just a few seconds

Even worse, athletes in these situations are not only subject to human error, they are often subject to human bias as well. We have all seen countless questionable judgment calls based on national or political bias in too many events. As upsetting as these are to the spectator they are utterly heart breaking for the athletes involved.

Bringing Machine Intelligence to the Rescue

We already use technology to aid in places where events happen to quickly for humans to accurately perceive them. In racing (humans to horses, on land or water), we use photo-finish cameras to resolve which athlete has actually one when a finish is too close (or as happened this year, when there is actually a tie for the Gold Medal). In Gymnastics and Skating we allow judges to review slow motion cameras as part of their judging. In Fencing, we go one step further and equip athletes with electronic sensors to measure when a blade has touched a target area (or which touched first to resolve simultaneous touches).

It is time to go a few steps further and actually bring machine intelligence (machine vision + machine learning) to the stage to provide the same absolute scoring that photo-finish cameras bring. I am not advocating using machines to replace people for stylistic judging. However, it makes absolutely no sense to not use machines to detect and score absolutes such as:

  • A gymnast’s bent arms, separated knees or mixed tempo
  • Level of differentiation of a diver’s twist from 90°
  • The actual time a sprinter kicks off the blocks based a microphone’s detection of when the sound arrived
  • Detection of a skater’s under-rotated jump

Not only would this significantly reduce bias and error. It would actually be a great training tool. Just as advanced athletes today use sensors to measure performance and conditioning, they could use a similar approach to detect small errors and work to eliminate them earlier in training.

This is Now Possible

Just a few years ago, this was the stuff to science fiction. Today it is feasible. Half a dozen companies have developer self-driving cars equipped with sensors and machine learning programs to deal with conditions with much higher levels of variability than judging a 10-meter dive or Balance Beam program. However, one does not need to equip arenas with multiple cameras and LIDAR arrays. Researchers at DARPA have even moved down the direction of teaching robots to cook by having them review two-dimensional YouTube videos.

Similar approaches could be uses for “Scoring Computers.” If we wanted to go down the path of letting computer see exactly (and only) what humans can see we can go down the machine-learning route. First program the rules for scores and penalties. Then create training sets with identified scores and infractions to train a computer to detect penalties and score them as a judge would do—but with the aid of slow motion review in a laboratory without the pressure of on-the-spot judging on live TV. This would not remove the human, it would just let the human teach a computer to do something with higher accuracy and speed than a person could do in real-time.

If we wanted to go a step further, just as Fencing has done. We can add sensors to mix. A LIDAR array could measure exact motion (actually measuring that bent knee or over-rotation). Motion- capture (mo-cap) would make this accuracy even better. Both would also create amazing advanced sports training technology.

It’s More Feasible Then You May Think

All of this technology sounds pretty expensive: computers, sensors, data capture, programming, testing, verification, deployment, etc. However, it is not nearly as expensive and “sci-fi-ish” as one might think (or fear).

Tens of thousands of hours of video already exists to train computers to judge events (the same videos that judges, athletes and coaches review in training libraries—libraries even better than robo.watch). Computing time is getting cheaper every year thanks to Moore’s Law and public cloud computing. An abundant number of Open Source libraries for machine learning are available (some companies have opened proprietary libraries; others are offering Machine Learning-as-a-Service). There are now even low-cost LIDAR sensors available for less than $500 that can resolve distances of 1 cm or less (for $2,000 college programs and Tier I competitive venues can get sensors that resolve to 1 mm or less).

Given the millions of dollars poured into these sports (and the billions into transmission rights), it would not require an Apollo Program to build a pilot of this in time for the 2020 Olympics (or even 2018 Winter Olympics). Companies like Google and IBM likely donate some R&D to show off their capabilities. Universities like MIT, Carnegie Mellon, and Stanford are already putting millions of dollars in biomimetics, computer vision, and more. Even companies like ILM and Weta Digital might offer their mo-cap expertise as they would benefit from joint R&D. Thousands of engineers would likely jump in to help out via Kaggle Competitions and Hackathons as this would be really fun to create.

Some Interesting Side Benefits

There are benefits to technology outside of “just” providing more accurate judging and better training tools. This same technology could create amazing television that would enable spectators to better appreciate and understand these amazing sports. Yes, you could also add your Oculus Rift or similar AR technology to create some amazing immersive games (creating new sources of funding for organizations like the US Olympic Team or USA Gymnastics to help pay for athlete training).

Ten Tech Trends for Your 2012 New Year’s Resolutions List

Article first published as Ten Tech Trends for Your 2012 New Year’s Resolutions List on Technorati

BabyNewYearOne of the most exciting things about working in tech is using it to create new ways to work, play—and even live. We have seen many great technology innovations develop over the past few years. Over 2012, ten of them will complete the jump from “new concept” to “mainstream trend.” How many of them are your ready for?

1. Everything Will Be Portable. The move to portable computing (smartphones, tablets and ultrabooks) will accelerate. Thick laptops and—even worse—desktops will be a relic of the past (except for those with high-power computing needs). If you are not yet mobile- and portable-ready, you better get there very soon.

2. Augmented Reality Will Go Mainstream. Augmented Reality (AR) is no longer a science fiction concept. Smartphones and (especially) tablets are mass-market platforms for everyday augmented reality. We are already seeing the first applications at Tech Meetups, CES and more. At least three innovators will exploit this, gaining mainstream adoption, by the end of 2012.

3. Touch Will Be Ubiquitous. Over the past five years, capacitive touch interfaces have re-programmed how millions of us interact with technology. As more devices are now sold today with touch than without, it is time to begin optimizing your user interface and user experience for touch (instead of a two-button mouse and keyboard).

4. Voice Will Be Next. While the intuitiveness of touch is a leapfrog improvement over mouse-and-keyboard, it still ties up our hands. Voice-based interaction is where we need to go. Apple’s Siri began the move of voice-driven interaction into the mainstream. This year, we’ll see SDKs for iOS and Android that harness the creativity of thousands to explode use of voice.

5. Fat Will Be the New Thin. Over a decade ago, broadband Internet enabled browsers to replace thick client applications. Now, portable computing usage across low power, lossy networks (e.g., mobile, WiFi, Bluetooth) coupled with AppStore Model has brought locally installed apps back in vogue. Building web apps is not enough; you need AppStore apps too.

6. Location-based Privacy Will Be Solved. Over the last two years location-based services became really hot. Unfortunately location-related privacy issues became hot too. The move of these services into mainstream populations of tens of millions will expand anecdotal security scares into weekly news stories, forcing adoption of safer location-based privacy policies.

7. Cloud Will Be the New Norm. Cloud computing is no longer an “edge market.” It is now adopted by big enterprises, public sector agencies—and even consumer tech providers. The cost, convenience and flexibility advantages of cloud computing will make it too hard for everyone not to use—everyday—by the end of this year.

8. …So Will Twitter. While people still love to debate the reasons to use Twitter, everything from the Arab Spring to the Charlie Sheen Meltdown showed that Twitter is now a well-recognized media channel. #Election2012 will accelerate mainstream use of Twitter—with the same overwhelming intensity we have seen for years in “traditional” campaign advertising.

9. ‘Consumerization of IT’ Planned and Budgeted. Consumer tech has become so sophisticated (without sacrificing ease-of-use and intuitiveness) that we began last year to demand its use in the enterprise. 2012—the first year in which most enterprise budgets include planned projects to support the consumerization of IT—will both accelerate and “lock in” this new tech trend.

10. 2012 Will Be Declared the Begin of “The ‘Big Data’ Era.” This year we will see another 40% increase in data we need to manage. This growth, coupled with recent releases of enterprise-ready high-scale NoSQL products will begin adoption of this tech by the entire industry. Looking back, 2012 will represent the start of the global, cross-industry Big Data era.

If you haven’t started embracing these already, now is a great time to add them to your “2012 Technology New Year’s Resolution List.” Sponsor a few pilot projects in your enterprise. Buy one or two Post-CES products to help you work more efficiently at the office. Or—if you want to include the whole family—buy one to use while you shop online, watch TV or manage your household.