This project is a robust location intelligence tool built with Node.js, Express.js, Puppeteer, and Redis. Born from a real-world frustration with the Israeli "Mechir Lamishtaken" lotteries, it seamlessly converts legacy ITM government grid coordinates into universal WGS84 GPS formats. Furthermore, it leverages the Google Gemini API and OpenStreetMap to provide dynamic, AI-powered neighborhood insights.
Visit the live project: Project Converter Deployment
Note: The live deployment operates under free-tier quotas, including a 1k units/month limit on Browserless.io and standard Google AI Studio rate limits. If these quotas are exceeded, conversion or AI features may be temporarily unavailable.
Pro Tip: For an unrestricted and high-performance experience, it is recommended to run the project locally using your own API keys as detailed in the Getting Started section.
Table of Contents
This project is a sophisticated full-stack web application designed to help users instantly locate "Mechir Lamishtaken" (Apartment at a Discount) real estate projects. Users typically discover these projects on the official Ministry of Construction and Housing (MOCH) Projects List, which serves as the primary government platform for subsidized housing lotteries in Israel.
Acting as an essential companion tool to the government portal, this application allows users to take any simple project number, lottery ID, or official MOCH URL and instantly translate it into an explorable environment. The system performs real-time web scraping, geospatial coordinate conversion, and AI-driven environmental analysis. It covers advanced backend and frontend engineering concepts, including:
- Real-Time Web Scraping: Dynamically navigates government maps (GovMap) using headless browsers to extract hidden ITM coordinates.
- Geospatial Coordinate Conversion: Mathematically translates local Israeli Grid (ITM) coordinates into global GPS standards (WGS84) for seamless Google Maps integration.
- AI Neighborhood Insights: Generates smart, on-demand summaries of the area's education, public transport, and future development using AI and reverse geocoding.
- Asynchronous Task Queue: Manages heavy scraping requests sequentially with real-time UI status updates, preventing server overload during traffic spikes.
- Full-Stack Cancellation Mechanism: Allows users to abort operations mid-flight via an
AbortController, instantly killing background cloud browser processes to conserve resources. - Smart Caching & Security: Uses persistent in-memory caching to serve repeated queries in milliseconds, combined with a custom "cache-aware" rate limiter to prevent bot abuse.
- Internationalization (i18n): Features a dynamic, React-less vanilla JS frontend that seamlessly switches between Hebrew (RTL) and English (LTR) without page reloads.
- Zero-Cost Engineering: Creatively integrates multiple free-tier cloud services to run a heavy automation and AI backend with zero operational costs.
- Node.js: Core backend runtime for asynchronous task orchestration.
- Express.js: Framework for creating robust server routes, middleware, and API endpoints.
- Puppeteer & Browserless.io: Cloud-based headless browser automation with dynamic residential proxy toggling.
- Redis (ioredis): In-memory database used for high-speed, permanent caching of project coordinates.
- Proj4: Mathematical library for highly accurate spatial coordinate transformations (ITM to WGS84).
- Google Gemini API: Dual-model AI integration (Flash 2.5 & Flash-Lite 3.1) for web research, data structuring, and automated translation.
- OpenStreetMap (OSM): Reverse geocoding API to extract official physical addresses and prevent AI location hallucinations.
- Vanilla JavaScript & CSS3: High-performance, modular ES6 frontend architecture with custom CSS keyframe animations (No heavy frameworks used).
- Express Rate Limit: Security middleware specifically customized to bypass limits when serving cached data.
- Render & Uptime Robot: Deployment platform and monitoring tool combined to ensure 24/7 availability on a free-tier environment.
- Dynamic Extraction: Navigates complex government maps (GovMap) using Puppeteer to locate and extract hidden ITM coordinates from dynamically injected URLs.
- Cloud Browser Management: Offloads heavy headless browser execution to Browserless.io via WebSockets, completely bypassing local server resource limits.
- Mathematical Conversion: Translates local Israeli Grid (ITM) coordinates into global GPS standards (WGS84) using the Proj4 library.
- Interactive Maps: Seamlessly generates and embeds precise Google Maps and updated GovMap iframes based on the calculated data.
- Reverse Geocoding: Queries OpenStreetMap (OSM) to validate exact physical addresses before AI processing, ensuring zero "location hallucinations".
- Smart Area Summaries: Leverages Google Gemini API (Flash 2.5) to perform web research and return structured data about local education, public transport, and future urban development.
- Multi-Model Translation: Utilizes a secondary, lightweight model (Flash-Lite 3.1) to translate the Hebrew JSON data into English on the fly when required.
- Asynchronous Queue Service: Processes heavy scraping requests sequentially using a custom FIFO queue, preventing server crashes during traffic spikes while providing real-time position updates to the user.
- Mid-Flight Cancellation: Implements the
AbortControllerAPI across the full stack. If a user cancels, the system instantly terminates the active cloud browser tab and removes the request from the queue to conserve API credits.
- Persistent Redis Cache: Stores successful coordinates in an in-memory database. Subsequent identical requests bypass the queue entirely and resolve in under 50 milliseconds.
- Cache-Aware Rate Limiting: Customized security middleware that prevents bot abuse (max 15 requests/15 mins) but intelligently ignores requests that are served from the cache, protecting free-tier quotas.
- Zero-Cost Architecture: Creatively strings together Render's free tier, Browserless, and Uptime Robot to maintain a 24/7 heavy automation backend without incurring operational costs.
- Internationalization (i18n): A robust, React-less Vanilla JS frontend that dynamically switches text and document direction (RTL/LTR) between Hebrew and English without page reloads.
- Race Condition Prevention: Sophisticated client-side polling logic that manages asynchronous UI updates, clears phantom timers, and accurately displays queue progression.
- UI/UX Polish: Features smooth CSS keyframe animations, dynamic loading overlays for map iframes, and FOUC (Flash of Unstyled Content) prevention techniques.
By exploring this project, you will learn how to:
- Architect a Zero-Cost Cloud Infrastructure: Creatively integrate free-tier services (Render, Browserless.io, Uptime Robot) to run heavy background workloads without incurring operational costs.
- Orchestrate Cloud Automation: Master headless browser manipulation (Puppeteer) over WebSockets to extract data from dynamically rendered and heavily obfuscated websites.
- Manage Asynchronous Workloads: Build and implement a custom FIFO task queue to protect server resources, prevent memory leaks, and manage concurrent traffic spikes.
- Implement Full-Stack Cancellation: Utilize the
AbortControllerAPI to seamlessly terminate active cloud browser processes mid-flight when a user aborts a request. - Integrate AI Responsibly: Combine Reverse Geocoding (OSM) with Large Language Models (Google Gemini) to anchor prompts in factual geographic data, effectively preventing AI hallucinations.
- Execute Geospatial Mathematics: Apply complex mathematical transformations using Proj4 to convert local grid coordinate systems (ITM) into global GPS standards (WGS84).
- Optimize Performance & Security: Leverage Redis for persistent caching and design custom, cache-aware rate limiters to protect API quotas and slash response times to under 50ms.
- Build Reactive Vanilla JS: Develop a high-performance, framework-less frontend featuring dynamic i18n (RTL/LTR switching) and advanced polling logic for state synchronization.
- Node.js (v18.x or later): Required for native
AbortControllerand modern ES modules support. - Redis: A local server or cloud instance (e.g., Upstash, Redis Cloud) for the caching layer.
- Browserless.io Account: An API token is required for headless browser automation in the cloud.
- Google AI Studio Account: A Gemini API key is required to power the Neighborhood Insights feature.
This project is deployed using Render. To maintain a zero-cost architecture while delivering high performance, the following cloud strategies were implemented:
- Bypassing Sleep Mode: Render's free tier normally spins down instances after 15 minutes of inactivity (causing 50+ second delays on cold starts). To solve this, the application is monitored by Uptime Robot, which pings a dedicated
/keep-aliveendpoint every 15 minutes to ensure the server remains awake and instantly accessible. - Cloud Browser Execution: Render's free tier lacks the CPU/RAM required to run headless Chrome locally. Therefore, the deployment is configured to route all Puppeteer traffic through Browserless.io via WebSockets, operating within their 1k units/month free tier quota.
Visit the live project: Project Converter Deployment
- Clone the repository
git clone [https://github.com/RamMichaeli17/Dira.git](https://github.com/RamMichaeli17/Dira.git)- Install NPM packages
npm install- Set up environment variables
- Create a
.envfile in the root directory and add your credentials/configurations:
# Port the server will run on
PORT=3000
# Node environment (development/production)
NODE_ENV=development
# Number of concurrent browsers in the pool (optional)
BROWSER_POOL_SIZE=1
# Run browsers in headless mode: true or false
HEADLESS_ON=false
# Your Google API key for general services (if separate from Gemini)
GOOGLE_API_KEY=YOUR_GOOGLE_API_KEY
# Redis connection URL (Example: redis://default:password@host:port)
REDIS_URL=redis://localhost:6379
# Browserless.io token if using remote Puppeteer
BROWSERLESS_TOKEN=YOUR_BROWSERLESS_TOKEN
# Whether to use Browserless: true or false
USE_BROWSERLESS=false
# Your Google Generative AI (Gemini) API Key
GEMINI_API_KEY=YOUR_GEMINI_API_KEY
# The Gemini model to use for neighborhood search and grounding
GEMINI_SEARCH_MODEL=gemini-2.5-flash
# The Gemini model to use for translation tasks
GEMINI_TRANSLATION_MODEL=gemini-3.1-flash-lite- Start the server
npm startUsing the Project Link Converter is highly intuitive. Here is the standard workflow:
- Find a Project: Locate an active lottery or project on the official Ministry of Construction and Housing (MOCH) website.
- Convert Location: Copy the project's official URL or its 3-5 digit ID number, paste it into the application's search bar, and click "Convert".
- View Interactive Maps: Watch the real-time queue status as the system extracts the ITM coordinates, converts them to WGS84, and generates precise Google Maps and GovMap iframes.
- AI Neighborhood Insights: Once the map loads, click the "✨ Analyze Project Environment with AI" button to receive an on-demand, detailed report covering local education, public transportation, and future urban development plans.
Distributed under the MIT License. See LICENSE.txt for more information.
Thanks to the following people who have contributed to this project:
 Ram Michaeli |
Ram Michaeli - ram153486@gmail.com
Project Link: https://github.com/RamMichaeli17/Dira
