AI / DSP Engineering
For software engineers, researchers and machine-learning practitioners. Study how neural audio systems represent sound, learn conditional relationships, generate audio, and are evaluated in production and research settings. If your aim is to mix or master a finished song in a DAW, take the Sound Engineering route instead.
Follow the path
Work through the material in three layers. The sequence moves from signal foundations to model design and then to production systems.
1. Represent audio
- Digital Audio Basics — establish the signal-level vocabulary used throughout the documentation.
- Music Representations — compare waveforms, spectrograms, symbolic formats, embeddings, and codec tokens.
- Signal Processing Basics — connect time-domain signals to frequency-domain analysis.
2. Model and train
- Transformers for Audio and Diffusion Models — examine core generation architectures.
- Dataset Curation — understand the data, rights, and labeling decisions that shape a model.
- Evaluation Metrics — combine computational measurements with structured listening tests.
3. Build a system
Apply the foundations through APIs and Integration, AI Music Agents, and Advanced Topics. Use Reliability and Sourcing when documenting claims and system behavior.
Choose by engineering task
| I need to… | Start here |
|---|---|
| Design an audio representation | Audio Embeddings and Neural Audio Codecs |
| Select a model architecture | Architecture |
| Prepare or augment training data | Training |
| Integrate a hosted model | APIs |
| Build a multi-model system | AI Music Agents |
| Fine-tune or control a model | Advanced Topics |
| Check terminology | Glossary |
These pages assume comfort with software concepts and introduce mathematics where it clarifies system behavior. Producer Guides prioritize creative decisions, while Sound Engineering covers session preparation, mix diagnostics, master delivery and provenance. The model-engineering curriculum here is intentionally separate.