You’ve probably seen it in the news. Different providers are accusing one another of distillation attacks. Distillation itself however is a legitimate means to post train a model. It’s also a great way to start to apply the path in a MoE (mixture of experts) architecture and put that into a smaller model. While there’s more to it, think of it as taking a strong model’s answers and using that as instructions for smaller models.
Distillation is not the only way you can fine tune your models. You can fine tune on your own data without having to run it on another model.
This is a more advanced thing to consider, but it can be very useful. The best analogy for this would be to think of a workflow you use AI, a naive implementation would use one of the Claudes for it (let’s say Sonnet). Then you can find an equivalent open model to replace that with (let’s say GLM5.2) - this alone drops your price by about 50%. Now you can find ways to improve GLM5.2 by tuning it, so for the same potential inference costs you can start to get better results.
Now apply that logic to every workflow you have, stable v1, cheaper but equivalently capable v2, same low cost but more capable v3.