Skip to content

modified save_site_database.py for nl and added new tests#139

Open
spyke7 wants to merge 1 commit intoopenclimatefix:mainfrom
spyke7:check_sum_nl
Open

modified save_site_database.py for nl and added new tests#139
spyke7 wants to merge 1 commit intoopenclimatefix:mainfrom
spyke7:check_sum_nl

Conversation

@spyke7
Copy link

@spyke7 spyke7 commented Jan 8, 2026

Pull Request

Description

This PR adds validation for Netherlands (NL) solar capacity data to prevent inconsistent (“funky”) regional capacity values from being saved to the site database.

A new helper function validate_nl_capacities checks, per timestamp, that the sum of regional capacities (regions 1–12) matches the national capacity (region 0) within a configurable tolerance (default 0.1%). If the validation fails, the save process is aborted to avoid persisting bad data.

The validation is integrated into save_generation_to_site_db() and only runs for NL data when the required columns (capacity_kw, region_id) are present.

Fixes #135

How Has This Been Tested?

Unit tests were added to cover:

  • Valid capacity data passing validation
  • Regional capacities that are too high or too low failing validation
  • Edge cases within the documented tolerance (~30 MW for NL)

Integration tests were added to ensure:

  • save_generation_to_site_db() aborts when NL capacity validation fails

  • Data is correctly saved when NL capacity validation passes

  • Yes

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

  • Yes

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

@peterdudfield
Copy link
Contributor

It looks like lots of these changes are formatting ones. Would you be able to split this PR into Formatting and NL tests?

@spyke7
Copy link
Author

spyke7 commented Jan 8, 2026

I did black . as written in the README.md, that's why the files are formatted

@spyke7
Copy link
Author

spyke7 commented Jan 8, 2026

Should I do it without doing the black . and push it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check sum of regional capacities against national before updating

2 participants