DynaMiCS: Fine-Tuning LLMs with Performance Constraints Using Dynamic Mixtures
Apple just solved the multi-domain fine-tuning problem that's been breaking every other lab's LLMs.
Why it matters
DynaMiCS addresses a critical bottleneck in LLM fine-tuning: how to improve performance on target domains without catastrophic forgetting on safety, instruction-following, and general knowledge. This is directly relevant to enterprises fine-tuning models on proprietary data.
The key facts
11 to knowApple research paper on constrained optimization for multi-domain fine-tuning
DynaMiCS uses dynamic mixture approach with domain-specific probing runs
Solves capability preservation problem (safety, instruction-following, general knowledge)
Casts fine-tuning as constrained optimization problem
Published on Apple ML Research (machinelearning.apple.com)
July 2026 publication date
Apple research on constrained optimization for multi-domain fine-tuning
Method: dynamic mixture optimizer using domain-specific probing runs
Solves preservation of general knowledge, instruction following, and safety during fine-tuning
Addresses limitation of existing fixed heuristic and adaptive data mixing strategies
Published July 2026 on Apple ML Research
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Multi-domain fine-tuning of large language models requires improving performance on target domains while preserving performance on constrained domains, such as general knowledge, instruction following, or safety evaluations. Existing data mixing strategies rely on fixed heuristics or adaptive rules…