Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
6772f2f
fix: move foreman to global gems to fix startup crash (#1971)
rtuszik Nov 25, 2025
ac9b668
Update exporting code to stream points data to file in batches to red…
Freika Nov 27, 2025
cebbc28
Update changelog
Freika Nov 29, 2025
4421a5b
Feature/maplibre frontend (#1953)
Freika Dec 6, 2025
97d52f9
Update maplibre controller
Freika Dec 6, 2025
9fb4bc5
Update changelog
Freika Dec 6, 2025
c5ef4d3
Remove some console.log statements
Freika Dec 6, 2025
672c308
Merge branch 'master' into dev
Freika Dec 6, 2025
2debcd8
Pull only necessary data for map v2 points
Freika Dec 6, 2025
9661e8e
Feature/raw data archive (#2009)
Freika Dec 7, 2025
cdf1428
Merge remote-tracking branch 'origin' into dev
Freika Dec 7, 2025
a7f77b0
Set raw_data to an empty hash instead of nil when archiving
Freika Dec 7, 2025
913d608
Fix storage configuration and file extraction
Freika Dec 8, 2025
6cc8ba0
Merge branch 'master' into dev
Freika Dec 8, 2025
1c9843d
Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation (#2018)
Freika Dec 8, 2025
9ac4566
Use user timezone to show dates on maps (#2020)
Freika Dec 8, 2025
516cfab
Fix/pre epoch time (#2019)
Freika Dec 8, 2025
c6d09c3
Update redis client configuration to support unix socket connection
Freika Dec 8, 2025
bb980f2
Update changelog
Freika Dec 8, 2025
8af032a
Fix kml kmz import issues (#2023)
Freika Dec 9, 2025
2a4ed8b
Implement moving points in map v2 and fix route rendering logic to ma…
Freika Dec 10, 2025
353837e
fix(maplibre): update date format to ISO 8601 (#2029)
rtuszik Dec 11, 2025
88f5e2a
Add verification step to raw data archival process (#2028)
Freika Dec 13, 2025
acee848
Eliminate zip-bomb risk
Freika Dec 13, 2025
b81d258
Fix potential memory leak in js
Freika Dec 13, 2025
b7f0b7e
Return .keep files
Freika Dec 13, 2025
1090bcd
Use Toast instead of alert for notifications
Freika Dec 13, 2025
f2d96e5
Add help section to navbar dropdown
Freika Dec 14, 2025
0b6149b
Update changelog
Freika Dec 14, 2025
c1bb7f3
Remove raw_data_archival_job
Freika Dec 14, 2025
20a4553
Ensure file is being closed properly after reading in Archivable concern
Freika Dec 14, 2025
35995e7
Add composite index to stats table if not exists
Freika Dec 14, 2025
d40b2a1
Update changelog
Freika Dec 14, 2025
87baf8b
Update entrypoint to always sync static assets (not only new ones)
Freika Dec 16, 2025
9ef0da2
Add family layer to MapLibre maps (#2055)
Freika Dec 26, 2025
1471e4d
Update changelog
Freika Dec 26, 2025
04fbe4d
Merge remote-tracking branch 'origin' into dev
Freika Dec 26, 2025
d4e87ce
Return changelog
Freika Dec 26, 2025
ad385f4
Update changelog
Freika Dec 26, 2025
32f5d2f
Update tailwind file
Freika Dec 26, 2025
e12b45f
Bump sentry-rails from 6.0.0 to 6.1.0 (#1945)
dependabot[bot] Dec 26, 2025
da9742b
Bump turbo-rails from 2.0.17 to 2.0.20 (#1944)
dependabot[bot] Dec 26, 2025
ce74b3d
Bump webmock from 3.25.1 to 3.26.1 (#1943)
dependabot[bot] Dec 26, 2025
7347be9
Bump brakeman from 7.1.0 to 7.1.1 (#1942)
dependabot[bot] Dec 26, 2025
03697ec
Bump redis from 5.4.0 to 5.4.1 (#1941)
dependabot[bot] Dec 26, 2025
c9ba791
Put import deletion into background job (#2045)
Freika Dec 26, 2025
3c1d17b
fix null type error and update heatmap styling (#2037)
rtuszik Dec 26, 2025
f325fd7
Fix stats calculation to recursively reduce H3 resolution when too ma…
Freika Dec 26, 2025
9805c55
Validate trip start and end dates (#2066)
Freika Dec 26, 2025
fab0121
Update migration to clean up duplicate stats before adding unique index
Freika Dec 26, 2025
fe9d7d2
Merge remote-tracking branch 'origin' into dev
Freika Dec 26, 2025
3f436c1
Fix fog of war radius setting being ignored and applying settings cau…
Freika Dec 26, 2025
4be58d4
Update changelog
Freika Dec 26, 2025
573d527
Add Rack::Deflater middleware to config/application.rb to enable gzip…
Freika Dec 26, 2025
67d7123
Add composite index to points on user_id and timestamp
Freika Dec 26, 2025
a722e19
Deduplicte points based on timestamp brought to unix time
Freika Dec 26, 2025
9e933af
Fix/stats cache invalidation (#2072)
Freika Dec 27, 2025
e857f52
Add new indicies to improve performance and remove unused ones to opt…
Freika Dec 28, 2025
18b13fb
Add yearly digest (#2073)
Freika Dec 28, 2025
14a0bb6
Update changelog
Freika Dec 28, 2025
26062a1
Add RailsPulse (#2079)
Freika Dec 28, 2025
5455228
Bring points_count to integer
Freika Dec 28, 2025
2f5487c
Update migration and version
Freika Dec 30, 2025
7f27761
Update rubocop issues
Freika Dec 30, 2025
2f11003
Fix migrations and data verification to remove safety_assured blocks …
Freika Dec 30, 2025
8a1e42a
Update version
Freika Dec 30, 2025
b4c2def
Merge remote-tracking branch 'origin' into dev
Freika Dec 30, 2025
b037be3
Update calculation of time spent in a country for year-end digest ema…
Freika Jan 3, 2026
8ecb2e3
Bump trix from 2.1.15 to 2.1.16 in the npm_and_yarn group across 1 di…
dependabot[bot] Jan 3, 2026
24cbabf
Map v2 will no longer block the UI when Immich/Photoprism integration…
Freika Jan 3, 2026
aa3bf93
Bump rubocop-rails from 2.33.4 to 2.34.2 (#2080)
dependabot[bot] Jan 3, 2026
96a7888
Bump chartkick from 5.2.0 to 5.2.1 (#2081)
dependabot[bot] Jan 3, 2026
966dc01
Bump rubyzip from 3.2.0 to 3.2.2 (#2082)
dependabot[bot] Jan 3, 2026
a4b9ed1
Bump sentry-ruby from 6.0.0 to 6.2.0 (#2083)
dependabot[bot] Jan 3, 2026
594b7ff
Bump sidekiq from 8.0.8 to 8.1.0 (#2084)
dependabot[bot] Jan 3, 2026
ec524d6
Update digest calculation to use actual time spent in countries based…
Freika Jan 4, 2026
1dc31a6
Update Gemfile and Gemfile.lock to pin connection_pool and sidekiq ve…
Freika Jan 4, 2026
348bf96
Rework country tracked days calculation
Freika Jan 4, 2026
88ae7c4
Adjust calculate_duration_in_minutes to only count continuous presenc…
Freika Jan 4, 2026
d664a10
Move helpers for digest city progress to a helper method
Freika Jan 4, 2026
fc2707a
Implement globe projection option for Map v2 using MapLibre GL JS.
Freika Jan 4, 2026
6f8bdce
Update time spent calculation for country minutes in user digests
Freika Jan 4, 2026
03069ca
Stats are now calculated with more accuracy by storing total minutes …
Freika Jan 4, 2026
d4416ed
Merge branch 'master' into dev
Freika Jan 4, 2026
b392527
Add globe_projection setting to safe settings
Freika Jan 4, 2026
322ae31
Remove console.logs from most of map v2
Freika Jan 4, 2026
ce8a7cd
Implement some performance improvements and caching for various featu…
Freika Jan 7, 2026
6e9e023
Add immediate verification and count validation to raw data archiving…
Freika Jan 9, 2026
f4ce656
Move points creation logic from background jobs to service objects (#…
Freika Jan 11, 2026
c3c5410
Add tracks to map v2 (#2142)
Freika Jan 11, 2026
462f5f3
Merge remote-tracking branch 'origin' into dev
Freika Jan 11, 2026
4a9a873
Minor changes
Freika Jan 11, 2026
301c14d
Fix minor issues
Freika Jan 11, 2026
aec9df0
Do not store thumbnails in cache if there are errors in the response
ArnyminerZ Jan 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Repository Guidelines

## Project Structure & Module Organization
Dawarich is a Rails 8 monolith. Controllers, models, jobs, services, policies, and Stimulus/Turbo JS live in `app/`, while shared POROs sit in `lib/`. Configuration, credentials, and cron/Sidekiq settings live in `config/`; API documentation assets are in `swagger/`. Database migrations and seeds live in `db/`, Docker tooling sits in `docker/`, and docs or media live in `docs/` and `screenshots/`. Runtime artifacts in `storage/`, `tmp/`, and `log/` stay untracked.

## Architecture & Key Services
The stack pairs Rails 8 with PostgreSQL + PostGIS, Redis-backed Sidekiq, Devise/Pundit, Tailwind + DaisyUI, and Leaflet/Chartkick. Imports, exports, sharing, and trip analytics lean on PostGIS geometries plus workers, so queue anything non-trivial instead of blocking requests.

## Build, Test, and Development Commands
- `docker compose -f docker/docker-compose.yml up` — launches the full stack for smoke tests.
- `bundle exec rails db:prepare` — create/migrate the PostGIS database.
- `bundle exec bin/dev` and `bundle exec sidekiq` — start the web/Vite/Tailwind stack and workers locally.
- `make test` — runs Playwright (`npx playwright test e2e --workers=1`) then `bundle exec rspec`.
- `bundle exec rubocop` / `npx prettier --check app/javascript` — enforce formatting before commits.

## Coding Style & Naming Conventions
Use two-space indentation, snake_case filenames, and CamelCase classes. Keep Stimulus controllers under `app/javascript/controllers/*_controller.ts` so names match DOM `data-controller` hooks. Prefer service objects in `app/services/` for multi-step imports/exports, and let migrations named like `202405061210_add_indexes_to_events` manage schema changes. Follow Tailwind ordering conventions and avoid bespoke CSS unless necessary.

## Testing Guidelines
RSpec mirrors the app hierarchy inside `spec/` with files suffixed `_spec.rb`; rely on FactoryBot/FFaker for data, WebMock for HTTP, and SimpleCov for coverage. Browser journeys live in `e2e/` and should use `data-testid` selectors plus seeded demo data to reset state. Run `make test` before pushing and document intentional gaps when coverage dips.

## Commit & Pull Request Guidelines
Write short, imperative commit subjects (`Add globe_projection setting`) and include the PR/issue reference like `(#2138)` when relevant. Target `dev`, describe migrations, configs, and verification steps, and attach screenshots or curl examples for UI/API work. Link related Discussions for larger changes and request review from domain owners (imports, sharing, trips, etc.).

## Security & Configuration Tips
Start from `.env.example` or `.env.template` and store secrets in encrypted Rails credentials; never commit files from `gps-env/` or real trace data. Rotate API keys, scrub sensitive coordinates in fixtures, and use the synthetic traces in `db/seeds.rb` when demonstrating imports.
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).

# [0.37.3] - Unreleased

## Fixed

- Routes are now being drawn the very same way on Map V2 as in Map V1. #2132 #2086
- RailsPulse performance monitoring is now disabled for self-hosted instances. It fixes poor performance on Synology. #2139

## Changed

- Map V2 points loading is significantly sped up.
- Points size on Map V2 was reduced to prevent overlapping.
- Points sent from Owntracks and Overland are now being created synchronously to instantly reflect success or failure of point creation.

# [0.37.2] - 2026-01-04

## Fixed
Expand All @@ -12,6 +25,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- Time spent in a country and city is now calculated correctly for the year-end digest email. #2104
- Updated Trix to fix a XSS vulnerability. #2102
- Map v2 UI no longer blocks when Immich/Photoprism integration has a bad URL or is unreachable. Added 10-second timeout to photo API requests and improved error handling to prevent UI freezing during initial load. #2085

## Added
- In Map v2 settings, you can now enable map to be rendered as a globe.

# [0.37.1] - 2025-12-30
Expand Down
41 changes: 41 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,47 @@ bundle exec bundle-audit # Dependency security
- Respect expiration settings and disable sharing when expired
- Only expose minimal necessary data in public sharing contexts

### Route Drawing Implementation (Critical)

⚠️ **IMPORTANT: Unit Mismatch in Route Splitting Logic**

Both Map v1 (Leaflet) and Map v2 (MapLibre) contain an **intentional unit mismatch** in route drawing that must be preserved for consistency:

**The Issue**:
- `haversineDistance()` function returns distance in **kilometers** (e.g., 0.5 km)
- Route splitting threshold is stored and compared as **meters** (e.g., 500)
- The code compares them directly: `0.5 > 500` = always **FALSE**

**Result**:
- The distance threshold (`meters_between_routes` setting) is **effectively disabled**
- Routes only split on **time gaps** (default: 60 minutes between points)
- This creates longer, more continuous routes that users expect

**Code Locations**:
- **Map v1**: `app/javascript/maps/polylines.js:390`
- Uses `haversineDistance()` from `maps/helpers.js` (returns km)
- Compares to `distanceThresholdMeters` variable (value in meters)

- **Map v2**: `app/javascript/maps_maplibre/layers/routes_layer.js:82-104`
- Has built-in `haversineDistance()` method (returns km)
- Intentionally skips `/1000` conversion to replicate v1 behavior
- Comment explains this is matching v1's unit mismatch

**Critical Rules**:
1. ❌ **DO NOT "fix" the unit mismatch** - this would break user expectations
2. ✅ **Keep both versions synchronized** - they must behave identically
3. ✅ **Document any changes** - route drawing changes affect all users
4. ⚠️ If you ever fix this bug:
- You MUST update both v1 and v2 simultaneously
- You MUST migrate user settings (multiply existing values by 1000 or divide by 1000 depending on direction)
- You MUST communicate the breaking change to users

**Additional Route Drawing Details**:
- **Time threshold**: 60 minutes (default) - actually functional
- **Distance threshold**: 500 meters (default) - currently non-functional due to unit bug
- **Sorting**: Map v2 sorts points by timestamp client-side; v1 relies on backend ASC order
- **API ordering**: Map v2 must request `order: 'asc'` to match v1's chronological data flow

## Contributing

- **Main Branch**: `master`
Expand Down
6 changes: 5 additions & 1 deletion app/controllers/api/v1/overland/batches_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,13 @@ class Api::V1::Overland::BatchesController < ApiController
before_action :validate_points_limit, only: %i[create]

def create
Overland::BatchCreatingJob.perform_later(batch_params, current_api_user.id)
Overland::PointsCreator.new(batch_params, current_api_user.id).call

render json: { result: 'ok' }, status: :created
rescue StandardError => e
Sentry.capture_exception(e) if defined?(Sentry)

render json: { error: 'Batch creation failed' }, status: :internal_server_error
end

private
Expand Down
8 changes: 6 additions & 2 deletions app/controllers/api/v1/owntracks/points_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,13 @@ class Api::V1::Owntracks::PointsController < ApiController
before_action :validate_points_limit, only: %i[create]

def create
Owntracks::PointCreatingJob.perform_later(point_params, current_api_user.id)
OwnTracks::PointCreator.new(point_params, current_api_user.id).call

render json: {}, status: :ok
render json: [], status: :ok
rescue StandardError => e
Sentry.capture_exception(e) if defined?(Sentry)

render json: { error: 'Point creation failed' }, status: :internal_server_error
end

private
Expand Down
11 changes: 9 additions & 2 deletions app/controllers/api/v1/photos_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,16 @@ def thumbnail
private

def fetch_cached_thumbnail(source)
Rails.cache.fetch("photo_thumbnail_#{params[:id]}", expires_in: 1.day) do
Photos::Thumbnail.new(current_api_user, source, params[:id]).call
cached = Rails.cache.read("photo_thumbnail_#{params[:id]}")
return cached if cached

response = Photos::Thumbnail.new(current_api_user, source, params[:id]).call

if response.success?
Rails.cache.write("photo_thumbnail_#{params[:id]}", response, expires_in: 1.day)
end

response
end

def handle_thumbnail_response(response)
Expand Down
35 changes: 29 additions & 6 deletions app/controllers/api/v1/places_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ def index
include_untagged = tag_ids.include?('untagged')

if numeric_tag_ids.any? && include_untagged
# Both tagged and untagged: return union (OR logic)
tagged = current_api_user.places.includes(:tags, :visits).with_tags(numeric_tag_ids)
untagged = current_api_user.places.includes(:tags, :visits).without_tags
@places = Place.from("(#{tagged.to_sql} UNION #{untagged.to_sql}) AS places")
.includes(:tags, :visits)
# Both tagged and untagged: use OR logic to preserve eager loading
tagged_ids = current_api_user.places.with_tags(numeric_tag_ids).pluck(:id)
untagged_ids = current_api_user.places.without_tags.pluck(:id)
combined_ids = (tagged_ids + untagged_ids).uniq
@places = current_api_user.places.includes(:tags, :visits).where(id: combined_ids)
elsif numeric_tag_ids.any?
# Only tagged places with ANY of the selected tags (OR logic)
@places = @places.with_tags(numeric_tag_ids)
Expand All @@ -30,6 +30,29 @@ def index
end
end

# Support pagination (defaults to page 1 with all results if no page param)
page = params[:page].presence || 1
per_page = [params[:per_page]&.to_i || 100, 500].min

# Apply pagination only if page param is explicitly provided
if params[:page].present?
@places = @places.page(page).per(per_page)
end

# Always set pagination headers for consistency
if @places.respond_to?(:current_page)
# Paginated collection
response.set_header('X-Current-Page', @places.current_page.to_s)
response.set_header('X-Total-Pages', @places.total_pages.to_s)
response.set_header('X-Total-Count', @places.total_count.to_s)
else
# Non-paginated collection - treat as single page with all results
total = @places.count
response.set_header('X-Current-Page', '1')
response.set_header('X-Total-Pages', '1')
response.set_header('X-Total-Count', total.to_s)
end

render json: @places.map { |place| serialize_place(place) }
end

Expand Down Expand Up @@ -120,7 +143,7 @@ def serialize_place(place)
note: place.note,
icon: place.tags.first&.icon,
color: place.tags.first&.color,
visits_count: place.visits.count,
visits_count: place.visits.size,
created_at: place.created_at,
tags: place.tags.map do |tag|
{
Expand Down
8 changes: 5 additions & 3 deletions app/controllers/api/v1/points_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,11 @@ def create
def update
point = current_api_user.points.find(params[:id])

point.update(lonlat: "POINT(#{point_params[:longitude]} #{point_params[:latitude]})")

render json: point_serializer.new(point).call
if point.update(lonlat: "POINT(#{point_params[:longitude]} #{point_params[:latitude]})")
render json: point_serializer.new(point.reload).call
else
render json: { error: point.errors.full_messages.join(', ') }, status: :unprocessable_entity
end
end

def destroy
Expand Down
16 changes: 16 additions & 0 deletions app/controllers/api/v1/tracks_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# frozen_string_literal: true

class Api::V1::TracksController < ApiController
def index
tracks_query = Tracks::IndexQuery.new(user: current_api_user, params: params)
paginated_tracks = tracks_query.call

geojson = Tracks::GeojsonSerializer.new(paginated_tracks).call

tracks_query.pagination_headers(paginated_tracks).each do |header, value|
response.set_header(header, value)
end

render json: geojson
end
end
11 changes: 11 additions & 0 deletions app/controllers/api/v1/visits_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,17 @@
class Api::V1::VisitsController < ApiController
def index
visits = Visits::Finder.new(current_api_user, params).call

# Support optional pagination (backward compatible - returns all if no page param)
if params[:page].present?
per_page = [params[:per_page]&.to_i || 100, 500].min
visits = visits.page(params[:page]).per(per_page)

response.set_header('X-Current-Page', visits.current_page.to_s)
response.set_header('X-Total-Pages', visits.total_pages.to_s)
response.set_header('X-Total-Count', visits.total_count.to_s)
end

serialized_visits = visits.map do |visit|
Api::VisitSerializer.new(visit).call
end
Expand Down
41 changes: 28 additions & 13 deletions app/controllers/map/leaflet_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -41,19 +41,34 @@ def build_tracks
end

def calculate_distance
return 0 if @coordinates.size < 2

total_distance = 0

@coordinates.each_cons(2) do
distance_km = Geocoder::Calculations.distance_between(
[_1[0], _1[1]], [_2[0], _2[1]], units: :km
)

total_distance += distance_km
end

total_distance.round
return 0 if @points.count(:id) < 2

# Use PostGIS window function for efficient distance calculation
# This is O(1) database operation vs O(n) Ruby iteration
import_filter = params[:import_id].present? ? 'AND import_id = :import_id' : ''

sql = <<~SQL.squish
SELECT COALESCE(SUM(distance_m) / 1000.0, 0) as total_km FROM (
SELECT ST_Distance(
lonlat::geography,
LAG(lonlat::geography) OVER (ORDER BY timestamp)
) as distance_m
FROM points
WHERE user_id = :user_id
AND timestamp >= :start_at
AND timestamp <= :end_at
#{import_filter}
) distances
SQL

query_params = { user_id: current_user.id, start_at: start_at, end_at: end_at }
query_params[:import_id] = params[:import_id] if params[:import_id].present?

result = Point.connection.select_value(
ActiveRecord::Base.sanitize_sql_array([sql, query_params])
)

result&.to_f&.round || 0
end

def parsed_start_at
Expand Down
10 changes: 7 additions & 3 deletions app/controllers/stats_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,12 @@ def precompute_year_distances
end

def build_stats
current_user.stats.group_by(&:year).transform_values do |stats|
stats.sort_by(&:updated_at).reverse
end.sort.reverse
columns = %i[id year month distance updated_at user_id]
columns << :toponyms if DawarichSettings.reverse_geocoding_enabled?

current_user.stats
.select(columns)
.order(year: :desc, updated_at: :desc)
.group_by(&:year)
end
end
Loading