Compile is an AI-powered news and media aggregator that curates articles based on user preferences and interests. It's pretty full of features and is something I had a lot of fun building with two of my friends. At a glance we...

• implemented a custom Python web-scraping framework on top of Crawl4AI, Stagehand, (and more)
• built an ingestion pipeline with heirarchal clustering and theme-aware embeddings
• tuned, prompt-optimized and context-managed lots of LLMs for the summaries
• built a full-stack JavaScript app for users to read, configure and explore articles
• setup a dev-box on my PC to schedule and manage pipelines with Temporal
• deployed it live at compile-news.com with Vercel