Training CodeParrot 🦜 from Scratch
Not a ChatGPT clone. Hugging Face just open-sourced CodeParrot, a code generation model trained from scratch.
Why it matters
Demonstrates the feasibility of training specialized AI models outside of Big Tech, with implications for the open-source AI ecosystem and developer accessibility to code generation tools.
The key facts
11 to knowCodeParrot is an open-source code generation model
Trained from scratch by Hugging Face
Published December 8, 2021
Represents alternative to proprietary code AI models
Signals Hugging Face's commitment to democratizing AI model development
CodeParrot is GPT-2 sized architecture
Model trained from scratch (not fine-tuned)
Open-source release via Hugging Face
Published December 2021
Focused on code generation task
Demonstrates alternative to proprietary code AI models
Go to the source
Hugging Face Bloghuggingface.co