Complete Website Downloader 💾 (güvenli fork)

Download the complete source code of any website (including all assets) 🔨.

Bu fork'ta güvenlik sertleştirmeleri yapılmıştır (AhmadIbrahiim/Website-downloader tabanlı):

• Komut enjeksiyonu giderildi: child_process.exec (shell) yerine execFile + argüman dizisi; URL -- sonrası geçirilir.
• SSRF engeli: URL çekilmeden önce doğrulanır — yalnız http/https, ve özel/loopback/link-local/CGNAT/bulut-metadata (169.254.169.254) IP'lerine çözülen hedefler reddedilir (lib/validateUrl.js).
• Kaynak sınırları: wget --timeout/--tries/--quota=2g; bağlantı kopunca süreç SIGTERM ile öldürülür; kısmi indirmeler temizlenir.
• Testler: npm test (URL doğrulama + SSRF birim testleri).
• Yalnız güvenilir/yetkili kullanım içindir; kamuya açık deploy için ek olarak kimlik doğrulama + rate-limit gerekir.

👉 Live Demo: https://website-downloader.onrender.com

Description 📒

Website downloader works with wget and archiver to download all websites assets and compress then sends it back to the user through socket channel

wget params the being used

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org

Explanation of the various flags:

• --mirror – Makes (among other things) the download recursive.
• --convert-links – convert all the links (also to stuff like CSS stylesheets) to relative, so it will be suitable for offline viewing.
• --adjust-extension – Adds suitable extensions to filenames (html or css) depending on their content-type.
• --page-requisites – Download things like CSS style-sheets and images required to properly display the page offline.
• --no-parent – When recursing do not ascend to the parent directory. It useful for restricting the download to only a portion of the site

Deploy on cloud providers

How to run it 🤔

• git clone https://github.com/AhmadIbrahiim/Website-downloader.git
• cd Website-downloader
• $ npm install
• $ npm start
• http://localhost:3000/

How To Contribute:

• Open Issue(s) with any bugs you notice.
• Please create Pull Requests if you think it would be an added value towards our program.

Liked it ? You can buy a coffee:

Thank you,

Email: me@ahmed-ibrahim.com

https://www.ahmed-ibrahim.com