Improvements v0.1.0 #12
Loading…
Reference in a new issue
No description provided.
Delete branch "improvements-v0.1.0"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Recover the 3 currently failing active sources, fix Python Weekly excerpt quality, surface This Week in Rust entries individually, lift the main-page loading cap, and refresh every logo derivative.
Source health
docker-blog,last-week-in-aws, andunsloth-blogviaapp/backend/services/source_headers.py. Worker wraps the fetcher per-source so ETag/Last-Modified still work and the override stays scoped (the other 297 sources are untouched).error_class=post_fetch_errorinstead of a misleading success with zero candidates.Content quality
style,script,noscript,template,svg, andiframe. Fixes the Beehiiv inline-CSS leak (p span[style*="font-size"] { line-height: 1.6; }) showing up in snippets.html_this_week_in_rustadapter that parses each Atom issue HTML and emits one entry per kept anchor. KeepsUpdates from Rust Communitysub-sections (Official, Project/Tooling Updates, Observations/Thoughts, Walkthroughs, Research, Miscellaneous); drops Crate of the Week, Calls for Testing, CFP, Events, project PR list, jobs, and Quote of the Week.Loading
MAX_LIST_ITEMS100 -> 500 andPAGE_SIZE_MAX100 -> 500. Auto-load now streams past the old cap on home, topic, stream, and search pages.Logo refresh
scripts/build_icons.shregenerates every raster derivative (apple-touch-icon.png/.webp,brand-mark.webp,icon-192.png,icon-512.png,favicon.ico,og/tilens-card.png) fromapp/frontend/static/tilens.pngusingffmpeg+cwebp+magick. Open Graph card is now deliberately simple: warm background, logo on the left, wordmark + tagline on the right.CACHE_VERSIONbumps"2"->"3"so installed PWAs pick up the new assets.Tests
tests/test_source_headers.pyandtests/test_this_week_in_rust_adapter.py.tests/test_icon_assets.pylocks PNG / WebP / ICO dimensions and asserts the WebP outputs are non-animated (noANIM/ANMFchunks, sinceffmpeg's WebP muxer would emit an animated container).tests/test_build_icons_script.pychecks the rebuild script is executable, declaresffmpeg+cwebp, and references every documented output.tests/test_python_weekly_adapter.py,tests/test_unsloth_adapter.py,tests/test_worker_dispatch_adapters.py,tests/test_chunking.py,tests/test_shell.py,tests/test_catalog.py,tests/test_pwa.py.tests/test_icon_assets.pyuses stdlib-only PNG / WebP / ICO parsing - no new dependency.Recover the 3 currently failing active sources, fix Python Weekly excerpt quality, expand the timeline for This Week in Rust, and let the main page load past 100 items. WAF-safe request headers: - New app/backend/services/source_headers.py allowlists docker-blog, last-week-in-aws, and unsloth-blog. Each receives browser-like User-Agent, Accept, Accept-Language, and a homepage Referer. Worker wraps the fetcher per-source so the override stays scoped and does not affect transparent-bot traffic to the other 297 sources. The wrapper preserves caller headers so ETag / Last-Modified conditional requests keep working. Python Weekly excerpt quality: - _SectionedArticleExtractor now skips text inside style, script, noscript, template, svg, and iframe tags while collecting snippets. Beehiiv previously leaked inline CSS like 'p span[style*="font-size"] { line-height: 1.6; }' into excerpts. Unsloth post subfetch hardening: - _load() treats status >= 400 as a failed subfetch. If the sitemap succeeds but every blog post returns an unhealthy status, the adapter now reports error_class='post_fetch_error' instead of a misleading success with zero candidates. This Week in Rust individual items: - New app/backend/adapters/this_week_in_rust.py parses the Atom feed's per-issue HTML and emits one candidate per kept anchor. The adapter only surfaces 'Updates from Rust Community' sub-sections (Official, Project/Tooling Updates, Observations/Thoughts, Rust Walkthroughs, Research, Miscellaneous) and drops the Crate of the Week, Calls for Testing, CFP, Events, project PR list, jobs, and Quote of the Week noise. config/sources.csv switches this-week-in-rust from feed to html_this_week_in_rust. Pagination cap lifted: - MAX_LIST_ITEMS moves from 100 to 500 so auto-load keeps streaming on the home, topic, stream, and search pages. - PAGE_SIZE_MAX in queries/helpers.py mirrors the same bump so search results can paginate to the new envelope. - test_chunking and test_shell reproduce pagination past the old cap with realistic entry counts. Tests: - tests/test_source_headers.py: allowlist scoping, UA tagging, Referer override, header immutability. - tests/test_python_weekly_adapter.py: CSS-before-description and script-before-description regression tests. - tests/test_unsloth_adapter.py: post_fetch_error on full failure, healthy subfetches preserved when some fail. - tests/test_this_week_in_rust_adapter.py: section filtering, tracking stripping, host skipping, relative URL handling, dedupe, malformed feed safety, HTTP error propagation. - tests/test_worker_dispatch_adapters.py: This Week in Rust dispatch, WAF header injection for docker-blog, unsloth subfetch, and a non-allowlisted source remains unmodified. Verification: - uv run pytest tests/ -q: 691 passed (was 654 before this release). - uv run ruff check: clean. - uv run python scripts/audit_sources.py: 300 active, 15 unsupported, no duplicate active endpoints, no issues. Version bump: - app/__init__.py, pyproject.toml, package.json, package-lock.json, uv.lock -> 0.1.0.Swapped in the new logo, so every raster derivative needs to come along. scripts/build_icons.sh generates them from the master app/frontend/static/tilens.png using ffmpeg for PNG/OG layout and cwebp for the static WebP outputs; ImageMagick still produces the multi-size favicon.ico and stays optional (fallback copies a 32x32 PNG if missing). The Open Graph card is now deliberately simple: warm background, the master logo on the left, wordmark plus short tagline on the right. Service worker CACHE_VERSION bumps from '2' to '3' so installed PWAs refresh the cached logo on next visit. Adds tests/test_icon_assets.py with stdlib-only PNG/WebP/ICO dimension parsing so a forgotten rebuild fails CI. Updates tests/test_pwa.py to assert the new cache version. docs/operations.md now documents the rebuild command and the expected asset inventory. Verification: - bash scripts/build_icons.sh regenerates every derivative. - file app/frontend/static/{apple-touch-icon,brand-mark,icon-*,favicon}.* app/frontend/static/og/tilens-card.png confirms sizes. - uv run pytest tests/ -q -> 698 passed (was 691). - uv run ruff check -> clean.