kbanc85
9956c9dee4
Improve extraction scripts, validation, and documentation
...
Enhance article extraction and validation systems with better error handling, retry logic, and content cleaning.
Script improvements:
- Add retry logic with exponential backoff for API calls
- Implement 60-second timeout for Anthropic API requests
- Improve HTML cleaning with Substack-specific filters
- Remove navigation, footer, and UI noise from extracted content
- Better HTML entity decoding
Validation enhancements:
- Add duplicate URL detection across articles
- Validate topic assignments against known topics
- Enhanced error reporting with specific issue types
GitHub Actions:
- Add build output validation step
- Verify /out directory exists before PR creation
- Count generated HTML files for sanity check
Documentation updates:
- Update project stats (26 articles, ~30 pages)
- Document v1.2 improvements
- Add changelog entries
Type safety:
- Improve topics.ts type definitions
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 14:06:19 -05:00
kbanc85