Connect with us

Uncategorized

WaveOne aims to make video AI-native and turn streaming upside down

Published

on

Video has worked the same way for a long, long time. And because of its unique qualities, video has been largely immune to the machine learning explosion upending industry after industry. WaveOne hopes to change that by taking the decades-old paradigm of video codecs and making them AI-powered — while somehow avoiding the pitfalls that would-be codec revolutionizers and “AI-powered” startups often fall into.

The startup has until recently limited itself to showing its results in papers and presentations, but with a recently raised $6.5M seed round, they are ready to move towards testing and deploying their actual product. It’s no niche: video compression may seem a bit in the weeds to some, but there’s no doubt it’s become one of the most important processes of the modern internet.

Here’s how it’s worked pretty much since the old days when digital video first became possible. Developers create a standard algorithm for compressing and decompressing video, a codec, which can easily be distributed and run on common computing platforms. This is stuff like MPEG-2, H.264, and that sort of thing. The hard work of compressing a video can be done by content providers and servers, while the comparatively lighter work of decompressing is done on the end user’s machines.

This approach is quite effective, and improvements to codecs (which allow more efficient compression) have led to the possibility of sites like YouTube. If videos were 10 times bigger, YouTube would never have been able to launch when it did. The other major change was beginning to rely on hardware acceleration of said codecs — your computer or GPU might have an actual chip in it with the codec baked in, ready to perform decompression tasks with far greater speed than an ordinary general-purpose CPU in a phone. Just one problem: when you get a new codec, you need new hardware.

But consider this: many new phones ship with a chip designed for running machine learning models, which like codecs can be accelerated, but unlike them the hardware is not bespoke for the model. So why aren’t we using this ML-optimized chip for video? Well, that’s exactly what WaveOne intends to do.

I should say that I initially spoke with WaveOne’s cofounders, CEO Lubomir Bourdev and CTO Oren Rippel, from a position of significant skepticism despite their impressive backgrounds. We’ve seen codec companies come and go, but the tech industry has coalesced around a handful of formats and standards that are revised in a painfully slow fashion. H.265, for instance, was introduced in 2013, but years afterwards its predecessor, H.264, was only beginning to achieve ubiquity. It’s more like the 3G, 4G, 5G system than version 7, version 7.1, etc. So smaller options, even superior ones that are free and open source, tend to get ground beneath the wheels of the industry-spanning standards.

This track record for codecs, plus the fact that startups like to describe practically everything is “AI-powered,” had me expecting something at best misguided, at worst scammy. But I was more than pleasantly surprised: In fact WaveOne is the kind of thing that seems obvious in retrospect and appears to have a first-mover advantage.

The first thing Rippel and Bourdev made clear was that AI actually has a role to play here. While codecs like H.265 aren’t dumb — they’re very advanced in many ways — they aren’t exactly smart, either. They can tell where to put more bits into encoding color or detail in a general sense, but they can’t, for instance, tell where there’s a face in the shot that should be getting extra love, or a sign or trees that can be done in a special way to save time.

But face and scene detection are practically solved problems in computer vision. Why shouldn’t a video codec understand that there is a face, then dedicate a proportionate amount of resources to it? It’s a perfectly good question. The answer is that the codecs aren’t flexible enough. They don’t take that kind of input. Maybe they will in H.266, whenever that comes out, and a couple years later it’ll be supported on high-end devices.

So how would you do it now? Well, by writing a video compression and decompression algorithm that runs on AI accelerators many phones and computers have or will have very soon, and integrating scene and object detection in it from the get-go. Like Krisp.ai understanding what a voice is and isolating it without hyper-complex spectrum analysis, AI can make determinations like that with visual data incredibly fast and pass that on to the actual video compression part.

Image Credits: WaveOne

Variable and intelligent allocation of data means the compression process can be very efficient without sacrificing image quality. WaveOne claims to reduce the size of files by as much as half, with better gains in more complex scenes. When you’re serving videos hundreds of millions of times (or to a million people at once), even fractions of a percent add up, let alone gains of this size. Bandwidth doesn’t cost as much as it used to, but it still isn’t free.

Understanding the image (or being told) also lets the codec see what kind of content it is; a video call should prioritize faces if possible, of course, but a game streamer may want to prioritize small details, while animation requires yet another approach to minimize artifacts in its large single-color regions. This can all be done on the fly with an AI-powered compression scheme.

There are implications beyond consumer tech as well: A self-driving car, sending video between components or to a central server, could save time and improve video quality by focusing on what the autonomous system designates important — vehicles, pedestrians, animals — and not wasting time and bits on a featureless sky, trees in the distance, and so on.

Content-aware encoding and decoding is probably the most versatile and easy to grasp advantage WaveOne claims to offer, but Bourdev also noted that the method is much more resistant to disruption from bandwidth issues. It’s one of the other failings of traditional video codecs that missing a few bits can throw off the whole operation — that’s why you get frozen frames and glitches. But ML-based decoding can easily make a “best guess” based on whatever bits it has, so when your bandwidth is suddenly restricted you don’t freeze, just get a bit less detailed for the duration.

Example of different codecs compressing the same frame.

These benefits sound great, but as before the question is not “can we improve on the status quo?” (obviously we can) but “can we scale those improvements?”

“The road is littered with failed attempts to create cool new codecs,” admitted Bourdev. “Part of the reason for that is hardware acceleration; even if you came up with the best codec in the world, good luck if you don’t have a hardware accelerator that runs it. You don’t just need better algorithms, you need to be able to run them in a scalable way across a large variety of devices, on the edge and in the cloud.”

That’s why the special AI cores on the latest generation of devices is so important. This is hardware acceleration that can be adapted in milliseconds to a new purpose. And WaveOne happens to have been working for years on video-focused machine learning that will run on those cores, doing the work that H.26X accelerators have been doing for years, but faster and with far more flexibility.

Of course, there’s still the question of “standards.” Is it very likely that anyone is going to sign on to a single company’s proprietary video compression methods? Well, someone’s got to do it! After all, standards don’t come etched on stone tablets. And as Bourdev and Rippel explained, they actually are using standards — just not the way we’ve come to think of them.

Before, a “standard” in video meant adhering to a rigidly defined software method so that your app or device could work with standards-compatible video efficiently and correctly. But that’s not the only kind of standard. Instead of being a soup-to-nuts method, WaveOne is an implementation that adheres to standards on the ML and deployment side.

They’re building the platform to be compatible with all the major ML distribution and development publishers like TensorFlow, ONNX, Apple’s CoreML, and others. Meanwhile the models actually developed for encoding and decoding video will run just like any other accelerated software on edge or cloud devices: deploy it on AWS or Azure, run it locally with ARM or Intel compute modules, and so on.

It feels like WaveOne may be onto something that ticks all the boxes of a major b2b event: it invisibly improves things for customers, runs on existing or upcoming hardware without modification, saves costs immediately (potentially, anyhow) but can be invested in to add value.

Perhaps that’s why they managed to attract such a large seed round: $6.5 million, led by Khosla Ventures, with $1M each from Vela Partners and Incubate Fund, plus $650K from Omega Venture Partners and $350K from Blue Ivy.

Right now WaveOne is sort of in a pre-alpha stage, having demonstrated the technology satisfactorily but not built a full-scale product. The seed round, Rippel said, was to de-risk the technology, and while there’s still lots of R&D yet to be done, they’ve proven that the core offering works — building the infrastructure and API layers comes next and amounts to a totally different phase for the company. Even so, he said, they hope to get testing done and line up a few customers before they raise more money.

The future of the video industry may not look a lot like the last couple decades, and that could be a very good thing. No doubt we’ll be hearing more from WaveOne as it migrates from lab to product.

Lyron Foster is a Hawaii based African American Musician, Author, Actor, Blogger, Filmmaker, Philanthropist and Multinational Serial Tech Entrepreneur.

Continue Reading
Comments

Uncategorized

Clubhouse announces plans for creator payments and raises new funding led by Andreessen Horowitz

Published

on

Buzzy live voice chat app Clubhouse has confirmed that it has raised new funding – without revealing how much – in a Series B round led by Andreessen Horowitz through the firm’s partner Andrew Chen. The app was reported to be raising at a $1 billion valuation in a report from The Information that landed just before this confirmation. While we try to track down the actual value of this round and the subsequent valuation of the company, what we do know is that Clubhouse has confirmed it will be introducing products to help creators on the platform get played, including subscriptions, tipping and ticket sales.

This funding round will also support a ‘Creator Grant Program’ being set up by Clubhouse, which will be used to “support emerging Clubhouse creators” according to the startup’s blog post. While the app has done a remarkable job attracting creator talent, including high-profile celebrity and political users, directing revenue towards creators will definitely help spur sustained interest, as well as more time and investment from new creators who are potentially looking to make a name for themselves on the platform, similar to YouTube and TikTok influencers before them.

Of course, adding monetization for users also introduces a method for Clubhouse itself to monetize. The platform is free to all users, and doesn’t yet offer any kind of premium plan or method of charging users, nor is it ad-supported. Adding ways for users to pay other users provides an opportunity for Clubhouse to retain a cut for its services.

The plans around monetization routes for creators appear to be relatively open-ended at this point, with Clubhouse saying it’ll be launching “first tests” around each of the three areas it mentions (tipping, tickets and subscriptions) over the “next few months.” It sounds like these could be similar to something like a Patreon built right into the platform. Tickets are a unique option that would go well with Clubhouse’s more formal roundtable discussions, and could also be a way that more organizations make use of the platform for hosting virtual events.

The startup also announced that it will be starting work on its Android app (it’s been iOS only for now) and that it will also invest in more backend scaling to keep up with demand, as well as support team growth and tools for detecting and prevuing abuse. Clubhouse has come under fire for its failure in regards to moderation and prevention of abuse in the past, so this aspect of its product development will likely be closely watched. The platform will also see changes to discovery aimed at surfacing relevant users, groups (‘clubs’ in the app’s parlance) and rooms.

During a regular virtual town hall the app’s founders host on the platform, CEO Paul Davison revealed that Clubhouse now has 2 million weekly active users. It’s also worth noting that Clubhouse says it now has “over 180 investors” in the company, which is a lot for a Series B – though many of those are likely small, independent investors with very little stake.

Continue Reading

Uncategorized

SpaceX sets new record for most satellites on a single launch with latest Falcon 9 mission

Published

on

SpaceX has set a new all-time record for the most satellites launched and deployed on a single mission, with its Transporter-1 flight on Sunday. The launch was the first of SpaceX’s dedicated rideshare missions, in which it splits up the payload capacity of its rocket among multiple customers, resulting in a reduced cost for each but still providing SpaceX with a full launch and all the revenue it requires to justify lauding one of its vehicles.

The launch today included 143 satellites, 133 of which were from other companies who booked rides. SpaceX also launched 10 of its own Starlink satellites, adding to the already more than 1,000 already sent to orbit to power SpaceX’s own broadband communication network. During a launch broadcast last week, SpaceX revealed that it has begun serving beta customers in Canada and is expanding to the UK with its private pre-launch test of that service.

Customers on today’s launch included Planet Labs, which sent up 48 SuperDove Earth imaging satellites; Swarm, which sent up 36 of its own tiny IoT communications satellites, and Kepler, which added to its constellation with eight more of its own communication spacecraft. The rideshare model that SpaceX now has in place should help smaller new space companies and startups like these build out their operational on-orbit constellations faster, complementing other small payload launchers like Rocket Lab, and new entrant Virgin Orbit, to name a few.

This SpaceX launch was also the first to deliver Starlink satellites to a polar orbit, which is a key part of the company’s continued expansion of its broadband service. The mission also included a successful landing and recovery of the Falcon 9 rocket’s first-stage booster, the fifth for this particular booster, and a dual recovery of the fairing halves used to protect the cargo during launch, which were fished out of the Atlantic ocean using its recovery vessels and will be refurbished and reused.

Continue Reading

Uncategorized

Watch SpaceX’s first dedicated rideshare rocket launch live, carrying a record-breaking payload of satellites

Published

on

 

SpaceX is set to launch the very first of its dedicated rideshare missions – an offering it introduced in 2019 that allows small satellite operators to book a portion of a payload on a Falcon 9 launch. SpaceX’s rocket has a relatively high payload capacity compared to the size of many of the small satellites produced today, so a rideshare mission like this offers smaller companies and startups a chance to get their spacecraft in orbit without breaking the bank. Today’s attempt is scheduled for 10 AM EST (7 AM PST) after a first try yesterday was cancelled due to weather. So far, weather looks much better for today.

The cargo capsule atop the Falcon 9 flying today holds a total of 143 satellites according to SpaceX, which is a new record for the highest number of satellites being launched on a single rocket – beating out a payload of 104 spacecraft delivered by Indian Space Research Organization’s PSLV-C37 launch back in February 2017. It’ll be a key demonstration not only of SpaceX’s rideshare capabilities, but also of the complex coordination involved in a launch that includes deployment of multiple payloads into different target orbits in relatively quick succession.

This launch will be closely watched in particular for its handling of orbital traffic management, since it definitely heralds what the future of private space launches could look like in terms of volume of activity. Some of the satellites flying on this mission are not much larger than an iPad, so industry experts will be paying close attention to how they’re deployed and tracked to avoid any potential conflicts.

Some of the payloads being launched today include significant volumes of startup spacecraft, including 36 of Swarm’s tiny IoT network satellites, and eight of Kepler’s GEN-1 communications satellites. There are also 10 of SpaceX’s own Starlink satellites on board, and 48 of Planet Labs’ Earth-imaging spacecraft.

The launch stream above should begin around 15 minutes prior to the mission start, which is set for 10 AM EST (7 AM PST) today.

Continue Reading

Trending