Commit
perf: optimize backend processing for large sample libraries
Commit details
Commit notes
Key optimizations implemented:
1. Unified Processing Queue - Combined audio processing + analysis into single job - Downloads file ONCE from S3 instead of twice - ~50% reduction in S3 bandwidth and latency
2. Batch AI Tagging - Accumulates tagging requests and sends in batches (up to 10) - Single API call instead of 10 separate calls - ~80-90% reduction in AI API latency overhead
3. Quick Metadata Skip Logic - Loop detection now has early exit for metadata tags - Combined with existing BPM/key filename detection - Instant results for well-labeled samples (Splice, Loopmasters, etc.)
4. New unifiedProcessingQueue - Concurrency: 4 workers - Replaces parallel audio-processing + audio-analysis queues - Simpler job flow, better error handling
Expected performance improvement for 100-sample batch: - Before: ~10 minutes - After: ~2 minutes (5x faster)
Documentation: See PROCESSING_OPTIMIZATIONS.md for full analysis
Co-authored-by: armin.naimi <[email redacted]>
- Files changed
- 5
- Lines added
- +1,045
- Lines removed
- −58