Fun interview from Jack Lucas Caffrey!
Related links:
Get Started with Graph Paper
If you need some graph paper... you already know why you're here. incompetech has the best graph paper generators available, and they're all easy to use!
Get Started with Music
If you need free music for your YouTube videos where you get to keep the ad revenue...
If you need music for your film or video game...
If you need music for your presentation or commercial...
Adobe Speech Enhance Review
Adobe has recently released a new tool called Adobe Speech Enhance, a part of their Adobe Podcast suite of AI tools. This product enhances voice recordings, especially those captured under suboptimal conditions, by recreating the speaker’s voice. I assume.
Performance
It works.
Here is a recording I took from my iPhone:
And here is the output from Adobe:
How does that compare to full studio recordings?
This is a direct recording using a Heil PR40 in a treated room.
Ease of Use
To use the web app, simply drag in your audio file and wait for it to render. As of this writing, expect it to process about 10 seconds of audio in a minute.
Could not be easier. This is next-level user interface.
A Bad Recording Day
Use Cases
Rescuing Backup Recordings. Sometimes your primary recording device fails. When it does, you can use whatever backup audio from whatever mics happen to be around.
Cleaning Up Walk-and-Talks. If you record in a noisy environment with traffic, or other noise.
Lecture Rescue. Lectures are often known for having bad audio. This just fixes that.
Travel Recordings. Pro travel setups can be really annoying and heavy. New plan: Leave the setup at home and record remotely with your phone.
Remote Interviews. Conducting interviews over video conferencing software can result in varied audio quality due to different mics and environments. Adobe Speech Enhance can unify the sound quality for a more professional output. You don’t need to care if your guest has a great mic setup!
Public Speaking Events. This tool can clean up live event recordings for later broadcasts or archives.
Historical Archiving. Old audio recordings that suffer from degradation or noise could potentially be cleaned up for historical or educational purposes.
Original recording from the Hindenburg Disaster:
Processed recording from the Hindenburg Disaster:
Original recoding from FDR’s speech:
Processed recording from FDR’s speech:
Oooh… this is so calming. Maybe I’ll just rest my eyes for a mi-
Morning
Morning. The time of day when you can’t figure out how to get to your drive-thru coffee place because there’s new construction, and the sun is super low so you’re squinting at the signs trying to figure out if you are supposed to turn left here, or if you definitely should NOT turn left here because you will get 3 tires shredded by sharp metal chunks before you fall in a hole.
This piece of music is based on the old-timey concept of calm, beautiful, renewing mornings.
AI or Copyright. Choose One.
DC District Court Judge Beryl A. Howell ruled on Friday, August 18th, 2023, that works created by AI are not eligible for copyright protections. The Judge ruled copyrights belong to people. If this decision goes unchallenged and becomes the precedent, content creators will have serious decisions to make.
Example of non-copyrightable image
A content creator could get an AI writer to craft their story, but they won’t own the copyright on that story. If you ask an AI to analyze the world’s data and come up with the absolute best story, that story will be in the Public Domain. Anyone who wants to use it, can use it.
Given the producer’s love of copyright, that’s going to be a tough decision to make.
The currently striking Writers Guild of America just received the biggest piece of ammunition for their cause.
Media producers will, of course, want cheap, good writers. But if producers use AI, they will find themselves competing in the marketplace without the protection of copyright on their stories. That’s not to say that the media won’t be profitable. There are plenty of countries where counterfeiting and piracy are daily occurrences—India, Nigeria, and China all have high levels of piracy, yet they also produce a lot of content.
Even if you eliminate counterfeiting and piracy, copyright doesn’t grant profitability. Studios can be profitable with or without copyright protections, but they will have to do things differently. They may have to work in secret and employ rapid distribution models. They may need to invent a distribution model so innovative that it’s simply better than anything else. Or they could lie.
Another example of a non-copyrightable-eligible image
Who’s Going to Lie to Whom?
It may be obvious that your soundtrack was written by an AI, but dialogue is more difficult to prove. It’s colored by actors, directors, cinematographers, and editors. If I were a writer, I might choose not to tell the studio exactly HOW that dialogue came to be. If studios want copyright on everything, they will need to verify that their writers did NOT use AI. How? No idea.
Or, studios could hire human writers and keep doing what they’re doing. This may prove cost prohibitive.
Could this ruling be appealed or challenged in higher courts?
Yup! This was one ruling in one federal district court.
Studios may push to get alternate rulings in other courts. If this happens, and different courts have different rulings, this could become an issue for the Supreme Court to address.
Not sure what it is, but it is uncopyrightable.
The future is unclear, but I’m enthused about this ruling!
Vampire Meemaw, play my favorite tune!
Southern Gothic
Are you cool? Cool like a vampire planking in a graveyard in Alabama?
Planking is uncool… so, you gotta be really, REALLY cool to pull off this vibe.
Stomp and clap along to the cooperating banjos!
Dynamic Orchestration, AI, and Video Games
There is a lot of hoopla out there with AI generated music and virtual singers. Most folks think these will be used in studios to produce soundtracks, and they are right. For now.
Dynamic Orchestration
Right now, every playthough of a specific game has the same music. Same opening theme, same battle music, same ‘pause screen’ music. If 500,000 people are playing Baulder’s Gate – each of them are getting the same soundtrack.
With Dynamic Orchestration, each playthrough will have a unique soundtrack. In an RPG, you can use music for setting and mood. There should be a music difference between “campsite music for a party of 4”, and “campsite music for a party of 4 who just picked up a vampire helper… and the night may not end well.”
Human composers are finite. We can’t cover all possibilities. A machine can. In real time.
History
We’ve had hints of dynamic orchestration for a while. Changing tempo to match the situation is common (like when everything speeds up for the last lap on Mario Kart). You can also mute or mix different layers of a piece of music to go from calm to exciting (like in an open world RPG where if you wander to where there is nothing, the music starts becoming thinner, and if you wander closer to interesting things, the music picks up). These are fairly difficult to implement now. They require working in MIDI-like sections, where instead of producing one track, you must produce thousands of individual notes and the instructions on how to put those notes together.
Why isn’t this happening now?
As of August, 2023, there are still a few hurdles.
The audio quality is pretty low right now. Eventually, we will get 48khz resolution on the audio, but now we have to settle for a fraction of that. Every company that does AI audio is working toward this, it’s a matter of time.
The cost of running these models is comparatively quite high. To do dynamic composition, you would need to stream a lot of audio data from servers OR find efficiencies in the models to allow them to run on smaller, home-scale computing devices.
What this means for Composers
There are still some opportunities for composers, though the workload per-game will be decreased significantly. A composer may still come up with the overall music design and provide some examples of themes for different parts of the story. We won’t be doing 12-hour soundtracks anymore.
Won’t AI also be able to do the main themes?
Yes. Yes, it will.
Ethical Considerations
Many people think the AI will be reigned in by governmental mandates and lawsuits from artists. These actions will slow adoption, but they can’t stop it. There is already enough music in the Public Domain to train some frighteningly good audio AIs. We have access to tons of folk music, music by dead composers like Mozart and Tchaikovski, and music be people who WANT to share what they’ve done like me.
Supercharged Storytelling
As Candy Crush weaponized engagement with their incredible design, Dynamic Orchestration will be a major aspect of weaponizing emotional storytelling… if you want it to. I hope every experience has a settings slider to control the poignancy of the soundtrack. Sometimes… I just don’t want that much.
Timeline
<Speculation> There are still a few challenges to get this to market. I expect someone will have a streaming service capable of doing this for under $10 per hour in under a year. Maybe another year to get an optimized model that can run on a graphics card or TPU. So… about 2027 you’ll be seeing the games show up on Steam. </Speculation>
Going Beyond
Dynamic Orchestration has a lot of other capabilities that can be realized before real-time inclusion in video games:
- Bar and Restaurant background music: You can get an endless playlist of jazz standards that perfectly read the room and adjust dynamically to the goings-on there.
- TV/Film Scoring: A composer will be able to guide the Dynamic Orchestration into an excellent soundtrack pretty quickly. Record it once, and incorporate it into the film!
- Personal Soundtrack: While going on a walk, working out, or just cleaning your house, you’ll be able to have a dedicated soundtrack perfectly suiting to inspiring you! (It may also include some musical warnings if your family is coming over soon).