This Thursday just gone, Native Instruments held a UK national event in several key retailers, giving more detail about the Kontrol D2, and in particular the new Stems technology. The most excellent demo by NI’s Chad Carrier dug deeper into the new format, and showed the out of the box potential for Stems. And not being in front of a laptop or behind a lens meant that my mind was free to ponder the potential roadmap for what I consider to be an immensely important technological step for DJs.
REALLY BIG DISCLAIMER
I must stress that these are my incoherent ramblings honed into an ideas piece. I certainly don’t have any inside information, not have I been prompted by NI to write this. Indeed neither I or any of the DJWORX team have played with Stems files yet. And believe me we’ve asked.
So before reposting or sharing this article, please be aware that this is entirely my train of thought. It would be cool if Stems tech allowed the ideas I’m proposing to become a thing, but don’t go sharing this as some sort of official NI missive, because it absolutely isn’t. There is nothing to read between the lines. Clear?
STEMS BASICS
Just to cover this off again, a Stems file is the master track, plus four individual stems such as drums, bass, vocal, and lead. NI will be supporting this out of the box with Traktor 2.9 (or so we were told at the UK NI events), and the Kontrol S8 and Kontrol D2 are the ideal controllers for this format.
Bar a few notable exceptions however, most tracks aren’t produced with just four individual stems. Even music prefixed with minimal is often quite complex in its construction. So for Stems files to contain only four individual stems in total is probably leaving quite a lot of the individually crafted bits of original music out.
At this point, my brain went off on a sleep-deprived trip. Having been shown so marvellously what Stems can do, my grey matter got to work on what Stems might be able to do in the future.
DEFINE YOUR OWN STEMS FILE
My thinking is that a master stems file should be just that — all stems (not just four) used to create a track be supplied in one big file, which will offer the ability to create your own stem file with just four selected stems inside. The MP4 format is a container that can store considerably more than the four stem limit being proposed in this new format, so this idea is quite possible.
For example, how cool would it be to be able to create your own version of a track and be able to just turn off the drums leaving the rest of the track intact, and then layer another track’s drums over the top on another deck? Alternatively to be able to have instrumental versions of tracks by just turning off the vocal? Perhaps you’re happy to have vocals and drums in a single stem, or maybe very individual parts of a song and not the drums, bass, vocal, and hook.
Stems as they stand will be a click and load thing with no track prep needed, and as such will make them considerably more attractive to more people — it’s called plug and play for a reason. But this idea gives control to you, and for you to decide just how you want your Stems files to be. Choice is good people, even if it does come with the ball and chain of track preparation.
FULL SONGS WITHIN STEMS
Having established that each Stem file is four individual stems, my mind got to thinking about what can be done with that. So instead of individual stems, what about putting full tracks in each stem file?
Because a Stems file loads into a single deck and thus has a master BPM, ideally you would tweak them to be synced and the same BPM within the master Stems file, so that all the songs can be faded in and out at will without the need to manually beat match. Essentially, a Stems file used in this way has the potential to load 16 full songs into a Kontrol S8 at once, with the ability to apply effects and filters to each one. Or if you choose, a mixture of songs and stems.
Just so you know, you can already load full tracks into remix deck slots, so this idea isn’t exactly unproven. But having 4 track loading at once, already synced to each other running from a master BPM is very handy. Obviously the master track will be a cacophony of beat matched noise, but you’ve now got way more music loaded than before. In fact, 16 tracks can be an entire set to a lot of people. That’s a lot less time gazing into your laptop screen.
STEMMED SONGS WITHIN STEMS
Taking this principle just one step further, what about taking the individual stems and creating a single stem with the individual stems laid out one after another?
This could them allow you to fully load up a Stems file with four fully stemmed tracks into a single deck. And if the original stems files had more than the current limit of four stems, you could start getting really creative and complex.
Now there are some serious limitations on how useable this could be right now — for example, how do you find the start of each stem track other than via waveform on screen? Hot cues can be added to a track, but you only get eight per stem file, a seemingly arbitrary limitation that really needs to be ditched. All I see is a Launchpad or Push’s 64 buttons laid out in a 4 deck grid, each with 4 cue point per stem — that’s 16 tracks, all loaded, stemmed, and the start of each stem just a button push away.
Given that Stems is open source, it should be relatively easy for some clever coder to make the “songs in Stems” and “stemmed songs in Stems” a reality. Drag and drop files into… umm… stem cells? Yeah I went there.
A FURTHER LEAP — AUTOMATION
Having all this quite amazing new tech at your fingertips opens up a huge amount of potential for raw DJing, performing, and producing. But the only way of capturing all this is to record the master mix. But to me, it seems that being able to record the sequence of events is probably quite doable, allowing for editing in post. Sorry – camera head on there.
When Serato and Ableton announced their Bridge collaboration, the most exciting thing for me was the Mixtape idea, where all music and hardware actions could be recorded, and tweaked in Ableton Live. So it’s not much of a reach to work out that NI is perfectly capable of doing the same, and recording every fader move, EQ tweak, and effects parameter to a file. This can then be edited in the Maschine software, or if NI were so inclined, created an additional open source Stems mix format that everyone else could adopt, allowing DJs to use Trakor to mix, and any other suitable tool to edit.
Quite a reach there, but that would be a powerful move for NI. Sometimes it’s not about directly monetising and all about effecting change for the greater good. So I’m glad to see NI getting behind it, even if they’re not really making any money directly from it.
DEEJUCER? PROJAY?
If you’ve worked through the whole piece and not just looked at the pretty pictures, you’ll have worked out that most of this is a pipe dream, and one that requires varying amounts work from NI and others to make it happen, and work from you to create the experience you want. It’s also clear that this goes some way beyond playing song A to song B and back again. It definitely isn’t DJing as we know it, and has the potential to energise hardware thinking too.
Never before have the lines been so blurred. Stems are a natural extension for DJs, but also give producers a valid method of live performance, an area where Ableton Live definitely excels, but perhaps isn’t aimed quite so much at producers. So somewhere in the middle is a hybrid DJ/performer/producer type that has yet to be given a name. Or alternatively, why label it at all and just have a bloody good time doing it?
SUMMING UP
Stems is a bold initiative, and one that has the potential to make more sense for DJs than Remix Decks ever did, and affect real change, and possible even a revolution in software and hardware. But it will require a large scale signup from some big names, both in technology but more importantly from artists and labels. It won’t be enough that unknown artists get behind this format — it’s going to take some big names with big tunes to enable Stems to get traction. Maybe just a couple to start with, but without music that people know being available, everything I just postulated above will remain as nothing more than ideas. This I feel is the area that the Stems consortium really needs to work on.
Classic hip hop track as stems, im interested. Distintive grooves from other genres
(in the way girltalk used fugazi “waiting room” bass, for example), im interested. All those movie quotes we all love and wanted to scratch, but couldn’t because they had music behind them, im interested. Tbh, im really surprised we arent talking about stem acapellas for scratching
I just dont see how giving me access to vocals and grooves that arent special(in the sense that they have stood the test of time, or are really recognizable) serves the purpose of remixing….
Here’s another big topic that can be done with audio stems, Mark: Spatial Sound (a.k.a. surround sound).
Ever since Native Instruments and Dubseed announced their stem services, I’ve been working with OpenAL, 3D animation exports, and high order ambisonics exports again, because the final piece finally fell into place for this to happen… content.
Granted, this isn’t a project you can just waltz in on and become great overnight, but since the foundation is there, elements can be transformed over time (and space), which results in a deeper emotional experience producing a song in 3D, rather than the stereo format.
Currently, I’m looking at both the One DJ, and Ableton’s Max for Live as the audio platforms, and Maya as the 3D environment and source manipulation interface, but it doesn’t just click like Legos, there are some very complex porting and assignment issues to make this work.
Ultimately, this changes the game of how we experience music.
Have you looked into OpenFrameworks? If not, you should check it out. It is a simple and very powerful interface and a great way to integrate any kind of visual and audio content. It might be a nice piece of the puzzle for your project, :)
I will, thanks for the suggestion. I get the feeling, the project I just embarked on will require a lot more than I can do alone.
Maybe if we enrolle a workgroup… :rolleyes:
Not as much a workgroup, but it certainly would make sense to start a small company based on this project. I’m pretty sure there’s enough money out there to fund something like this.
… Off to write a white paper.
God article Mark. I’m glad to read the automation part from you because we discuss these concept in the past and I hope it goes even further including the turntable vectorized data (position, speed, direction… So scratch) not to “become a scratch robot” but yes to a full transcription methodology which make turntablism a complete “instrumentation”. Serato mixtape lacked, Flip seems go in this direction and NI had in the past something similar (NHL recorder).
Agreed too in the blurred lines about the name… Can I suggest Tooltablism? :}
Dubbing could be another one due the work of Masters like Mad Professor were in the same direction but with different (their contemporany old today) tools but in essence the same music principle rear the goal/feeling.
Avout the possibilities of multiple stems (or more than 4) and all e derivarive process you exposed I think it will go for the Maschine route and I feel some “strange” that we all talking only about audio when midi alongside audio (in clip format or karaoke) could be interesting too. If traktor or maschine (or the fusion of them in futurible new engagement) become a somekind of “live DAW with turntable control” (maybe it will be lost in the meantime who knows? :( ) midi and vst instruments will be “necessary” so I can imagine Traktor as superaudio samplersequencer and maschine as supermidi sequencersampler… It doesn’t makes sense for anyone outhere?
I like “Audio Alchemist.”
No midi love? ;(
Just kidding. Free human expressing themselves honestly throught music will be illustrative and mind blowing in equal parts.
Media Alchemist sounds good to me ;)
Most of your thoughts should end up fairly easy to do actually. Even if NI doesn’t technically ‘support’ it they have already stated it won’t be a closed format. That means software like FFMPEG will make it very easy to rip out stems or combine individual stems into new tracks. As a VJ, I already use it all the time with mp4s to rip out audio / metadata / subtitles, or to mux together multiple feeds into a single container or even to overlay PNGs over video to make 1/2ass fake projection mappings.
Basically, you will be able to do everything you imagined in your article. Off the top of my head, all of these actions are straight forward in FFMPEG already:
1. Take a purchased stem master track, and rip out the individual stems
2. Combine random stems from multiple tracks into a single mp4 container
3. Combine complete mp3 songs as individual stems into a single mp4 container
4. Make hour long mp4s with stem after stem of individual tracks in 4 separate layers
All of this will be pretty trivial with a little FFMPEG knowledge. As far as syncing the bpms of the stems, or keeping things in specific parts, that should be pretty easy to do with a metadata track.
======= end of audio portion, beginning of personal musings =======
Stems should create a very nice niche industry for DJ/VJ synchronisation. When I DJ and do my own visuals, I line out the audio into a raspberry Pi or a computer and use openframeworks to do beat detection so the videos / shaders react accordingly. With a stemmed MP4 container, I could add a subtitle track that had all the lyrics of the songs (if there were any), which could then be piped right into the video feed and played on screen. Or, just add song title / record title information. Or just put in any random text crap I wanted outputting to the video. Or I could set up one of the audio stems to be just noise to use as a glitch generator to overlay on video in real time. Or, setup metadata signals in the container that will automatically cue changes in the video loops or effects without any input from me when the tracks change.
It is actually pretty interesting to think about these possibilities, :D
This kind if approarch is the most interesting: metadata.
It could even permit create a timecode stem inside these files which could control other stems not only related to dvs software… Think in korg volcas, put a audio synced stem of their audio sync signal in your tracks and voilâ no more jitter between audio and external hardware with sample resolution and no extra processing on decoding midi (as example) data…
The possibilities are endless :)
Isn’t that what SMPTE time code was made for?
Yes but it was used mainly in video linear editing. Using it with “turntable” control makes it a bit interesting for realtime edition (live) and in the other hand the possibility of matching some hotcues (or clip tagging) makes it non linear and really useful for vjing (as stated earlier dj0le)
Yep the beauty of NI committing to MP4 is how versatile the container is. People usually just think of it as a ‘movie’ extension, but in reality, it’s more like a .zip or .rar file. You can literally embed any kind of data you want into an mp4 container, in individual / separate ‘tracks’.
The only difficulty I see is getting Ni to output the internal software data. It will be up to them whether or not they want to send it *(ie – fader / knob positions etc), but they haven’t shown much love to Midi, so I am not sure they will do it here either
If they use the same code for Maschine (which is almost a DAW), they can.
I really wish Atomix hadn’t decided to ditch the Timeline feature in VDJ 8 before it got released.
It could potentially have developed into a Mixtape-like feature – being able to record actions from the mixer and decks as well as the audio from each deck, and even the sampler, then being able to edit to perfection.
Now that Atomix is getting VDJ to work with everything with a USB port, perhaps the feature will return, along with recording hardware movements.
I did suggest to them that it could be released as an add-on (even if it was paid for) which could be installed by those who wanted it. I don’t consider it very fair for Atomix to just decide behind the scenes that nobody would use it. A lot of DJs were looking forward to VDJ 8 for that very feature.
I suspect the specification of four stems was not technical, rather conforming to the current NI product line. High bandwidth Internet is everywhere and disk storage is cheap so more than 4 stems would not be an economical issue.
I can see producers using NI stems in their own performances..but I just don’t think that popular/main/bigger whatever you want to call them…producers will want commercial sales of their stems to be manipulated by the general public just to make a couple more dollars a sale, especially in the dance/house music world..maybe the Hip Hop world will be different, but the majority of Hip Hop DJs use Serato..so we will have to see if Serato really adds the feature and puts its weight behind it… it will take a while to get a Serato product for sale that will use stems as well as the D2..and in the mean time they probably won’t say anything about the technology..why would they help NI sell D2’s and S8’s?.. and NI has no incentive to open the D2 for Serato/VDJ/Pioneer mapping too quickly..So it might be dead in the water for a while…Plus I think people are underestimating the connection between a producer and his/her work..they recorded it one way for reason and even remixes need to get approved by the artist/label before release…but it might blow up as tool for live performances and NI will probably sell a lot of D2’s as a Traktor Controller anyway..
RE recording: you used to be able to record control changes in Traktor but they removed the feature some time ago.
Losing this feature was one of the biggest losses to Traktor, in my opinion. I didn’t need any extra software to edit my mix, I could do it all in Traktor natively, and edit individual movements across an entire mix. It made so much sense, but I think it wasn’t used. I don’t want to drop my mixes into Maschine, I want to make changes right in Traktor.
Pushing the whole live remixing and mashup concept to the side, think how stems could effect normal 2-deck DJing. You could basically load songs on a deck 4-at-a-time. Less time staring at your laptop (serato face) searching for the right song to play next…you always have 3 options at ready. Or if you’re playing a planned out set, you could play for a whole hour only loading like 5 tracks. Stem prep could become the new playlist prep.
Also I wonder how the stems playback will be implemented. With beatjump, slip modes, etc. the playhead is already a fluid thing. What if you’re able to effect each track on a stem differently: one track looping, while another does hot cues, while you’re scratching one track and another plays normal. etc. Even better…if this could be used with something like Serato Flip, which already can record events quantized or not. Add velocity triggering to hot cues (Serato’s sampler already responds to velocity) and with a Pioneer DDJ-SP1 you basically have an MPC on each deck. Score!
I feel like you guys (author and commenter) are headed in the “wrong” direction here. I put “wrong” in quotes because I would NEVER make a blanket statement that artists “should” use technology the way it’s “supposed” to be used… all the great artistic innovations in tech have come from using technology the “wrong” way. That said, a lot of you seem exclusively interested in automating your sets within an inch of their lives. Not just automating programming (which can be desirable in moderation), but automating performance. That’s exactly what i dislike about Serato’s MIxtape… you’re recording things I would personally insist on doing live, in performance. I don’t think it’s cheating, necessarily; but I do think it’s cheap.
To me what’s most interesting about Stems is they could bring LIVE, HANDS-ON remixing to the dancefloor. MORE live, spontaneous performing is what we need right now, not less. 4 stems is the just the right amount to keep it simple (4 was good enough for King Tubby), and having them all synced lets you work those volume faders and EQs, “play” keylock on a keyboard or scratch individual stems in Flux mode, etc. to your heart’s content. (And yeah, FX too, even though Traktor’s effects are way too labor-intensive for me to have ever bothered with live.) And to keep that synced ease-of-use and live vibe intact, I suspect four separate songs won’t work as Stems. I bet they’re kept in sync beginning to end… you might not be able to play the beginning of one Stem out of the middle/ end of another.
Again, I would never tell anyone what they “should” do with tech, but I want to see what Stems can do as intended, before I look for ways to force even more performance to leave the DJ booth and bore jaded dancefloors even further. I do agree it would be a great boon for producers performing live, but I think a lot of you guys may be producers looking for more Live-PA-type features out of DJ software. And, fair or not, the current Live PA-ers who’ve invested hundreds-to-thousands of bucks in live gear are going to be looking down their noses at “everyone’s a producer now” PAs before you know it.