zimwright

This scraper is a very minimal tool used to download ZIMs from online URL and push them in the Zimfarm and then CMS and so on.

Installation

Most recommended solution to run the scraper is to use pre-built Docker container.

Prerequisites are:

• Docker (or Docker-based engine)
• amd64 architecture

Usage

# Get help
docker run -v output:/output ghcr.io/openzim/zimwright zimwright --help

# Create a ZIM for Europe
docker run -v output:/output ghcr.io/openzim/zimwright zimwright --download-from <zim_online_url>

Advanced installation

You can also use more advanced installation:

Build your own container

1. Clone the repository locally:

   git clone https://github.com/openzim/zimwright.git && cd zimwright

2. Build the image:

   docker build -t ghcr.io/openzim/zimwright .

Run the software locally using Hatch

1. Clone the repository locally:

   git clone https://github.com/openzim/zimwright.git && cd zimwright

2. Install Hatch:

   pip3 install hatch

3. Start a hatch shell to install software and dependencies in an isolated virtual environment.

   hatch shell

4. Run the zimwright command:

   zimwright --help

Developing

Use the commands below to set up the project once:

# Install hatch if it isn't installed already.
❯ pip install hatch

# Local install (in default env) / re-sync packages
❯ hatch run pip list

# Set-up pre-commit
❯ pre-commit install

The following commands can then be used to build and test the scraper:

# Show scripts
❯ hatch env show

# linting, formating, type checking
❯ hatch run qa:check-all
❯ hatch run qa:fix-all

# run tests
❯ hatch run test:run

# building packages
❯ hatch build

Contributing

This project adheres to openZIM's Contribution Guidelines.

This project has implemented openZIM's Python bootstrap, conventions and policies v2.0.0.

See details for contributions in CONTRIBUTING.md.