Rewrite PSN + CityHome scrapers, add price/m² map coloring, ratings system, and status dashboard
- Rewrite PSN scraper to use /api/units-list endpoint (single API call, no HTML parsing) - Fix CityHome scraper: GPS from multiple URL patterns, address from table cells, no 404 retries - Color map markers by price/m² instead of disposition (blue→green→orange→red scale) - Add persistent rating system (favorite/reject) with Flask ratings server and localStorage fallback - Rejected markers show original color at reduced opacity with 🚫 SVG overlay - Favorite markers shown as ⭐ star icons with gold pulse animation - Add "new today" marker logic (scraped_at == today) with larger pulsing green outline - Add filter panel with floor, price, hide-rejected controls and ☰/✕ toggle buttons - Add generate_status.py for scraper run statistics and status.html dashboard - Add scraped_at field to all scrapers for freshness tracking - Update run_all.sh with log capture and status generation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,7 @@ WORKDIR /app
|
||||
|
||||
COPY scrape_and_map.py scrape_realingo.py scrape_bezrealitky.py \
|
||||
scrape_idnes.py scrape_psn.py scrape_cityhome.py \
|
||||
merge_and_map.py regen_map.py run_all.sh ./
|
||||
merge_and_map.py regen_map.py run_all.sh ratings_server.py ./
|
||||
|
||||
COPY build/crontab /etc/crontabs/root
|
||||
COPY build/entrypoint.sh /entrypoint.sh
|
||||
@@ -18,7 +18,7 @@ RUN chmod +x /entrypoint.sh run_all.sh
|
||||
|
||||
RUN mkdir -p /app/data
|
||||
|
||||
EXPOSE 8080
|
||||
EXPOSE 8080 8081
|
||||
|
||||
HEALTHCHECK --interval=60s --timeout=5s --start-period=300s \
|
||||
CMD wget -q -O /dev/null http://localhost:8080/ || exit 1
|
||||
|
||||
@@ -6,7 +6,7 @@ DATA_DIR="/app/data"
|
||||
# Create symlinks so scripts (which write to /app/) persist data to the volume
|
||||
for f in byty_sreality.json byty_realingo.json byty_bezrealitky.json \
|
||||
byty_idnes.json byty_psn.json byty_cityhome.json byty_merged.json \
|
||||
mapa_bytu.html; do
|
||||
mapa_bytu.html ratings.json; do
|
||||
# Remove real file if it exists (e.g. baked into image)
|
||||
[ -f "/app/$f" ] && [ ! -L "/app/$f" ] && rm -f "/app/$f"
|
||||
ln -sf "$DATA_DIR/$f" "/app/$f"
|
||||
@@ -18,5 +18,8 @@ crond -b -l 2
|
||||
echo "[entrypoint] Starting initial scrape in background..."
|
||||
bash /app/run_all.sh &
|
||||
|
||||
echo "[entrypoint] Starting ratings API server on port 8081..."
|
||||
DATA_DIR="$DATA_DIR" python3 /app/ratings_server.py &
|
||||
|
||||
echo "[entrypoint] Starting HTTP server on port 8080..."
|
||||
exec python3 -m http.server 8080 --directory "$DATA_DIR"
|
||||
|
||||
Reference in New Issue
Block a user