Feature request: Auto sync

[odilf]

Hello, I just started using taskwarrior and I really like!

I think it would be a good idea to have an option to be able to auto-sync tasks. Right now it does say when syncing is needed, but you have to do it manually which seems clunky (i.e., if you forget to do it and you don't have that device available your tasks are stale) and you still have to remember to sync yourself to get changes from other places.

Proposed solution

Add something like a rc.sync.auto option (open to bikeshed) to enable autosync. Maybe it would make sense to also add something like rc.sync.auto.modify-debounce and rc.sync.auto.read-debounce, to be able to configure some debounce timing to not do an unnecessary amount of requests (even though it's generally very little data).

Current alternatives

Right now I'm using a couple of hooks to sync on launch and on modify:

FLAGFILE="/tmp/taskwarrior-needs-sync"
LAST_SYNC="/tmp/taskwarrior-last-sync"
SYNC_INTERVAL=300  # 5 minutes

# Only sync if enough time has passed
time_passed=$(($(date +%s) - $(date -r "$LAST_SYNC" +%s 2>/dev/null || echo 0)))
if [ $time_passed -lt $SYNC_INTERVAL ]; then
  exit 0
fi

/nix/store/h3l2d5mmijwffnmpq3fw598x55d6x6cn-taskwarrior-3.4.1/bin/task rc.hooks=off sync > /dev/null 2>&1
touch "$LAST_SYNC"
exit 0

FLAGFILE="/tmp/taskwarrior-needs-sync"
PIDFILE="/tmp/taskwarrior-sync.pid"

# Echo tasks
read original_task
read added_task
echo "$added_task"

# Mark that we need a sync
touch "$FLAGFILE"

# Exit if sync process already running
if [ -f "$PIDFILE" ] && kill -0 $(cat "$PIDFILE") 2>/dev/null; then
  exit 0
fi

# Start background sync process
nohup /nix/store/ndyhaygmwxljqqssbcqz7ir0v5b7cjm9-bash-interactive-5.3p3/bin/bash -c '
  echo $$ > "'"$PIDFILE"'"
  sleep 5
  if [ -f "'"$FLAGFILE"'" ]; then
    rm "'"$FLAGFILE"'"
    '"/nix/store/h3l2d5mmijwffnmpq3fw598x55d6x6cn-taskwarrior-3.4.1/bin/task"' rc.hooks=off sync
  fi
  rm -f "'"$PIDFILE"'"
' > /dev/null 2>&1 </dev/null &

exit 0

but they seems to sometimes make task take waaay too long, it seems you have to use nohup and sometimes they seem to not work . In general I think auto syncing is a desirable feature to have built-in for new users.

I'm not too experienced with C++, but if this is something the team wants I'm willing to give a shot at doing a PR :)

[djmitche]

The general recommendation is to use cron to schedule sync's. If I recall, we discussed syncing automatically early on and decided against it. @ryneeverett do you recall more details?

The long delay on running task might be due to generation of a snapshot, which is particularly slow if TC is built in the debug profile -- see GothenburgBitFactory/taskchampion#593.

[ryneeverett]

In #3118 we experimented with combining automatic syncing with automatic garbage collection but we eventually realized that combining these features didn't make any sense. I don't think we concluded that automatic sync is an inherently bad idea.

That said, it isn't quite clear to me that there is a good way to integrate automatic sync with taskwarrior cli. Would it launch a background task like in the example? Is that something we can do reliably in a cross-platform compatible way? Is that the behavior users would expect and want?

An incremental step might be to offer to run sync if you haven't done so for a while with a [y/n] prompt in the cli.

[odilf]

If I recall, we discussed syncing automatically early on and decided against it.

Ah my bad, I did a cursory search but missed that discussion.

The general recommendation is to use cron to schedule sync's.

I though of that but it just seems like the wrong tool, since you know exactly when you need to sync: when modifying tasks and when viewing tasks if you haven't synced in a while.

Also, I'm on a mac where it's a bit more of a hassle to set up these things (incidentally, I'm using nix and home-manager which has an option for a cron job, but only for linux :P)

In #3118 we experimented with combining automatic syncing with automatic garbage collection but we eventually realized that combining these features didn't make any sense. I don't think we concluded that automatic sync is an inherently bad idea.

I think it does make sense, especially as an option. Of course it can be documented that, for instance, undo will basically not work. I'm happy with that tradeoff, and I think it's nice to have it as an option because even if it's not perfect there is definitely some use to auto sync.

That said, it isn't quite clear to me that there is a good way to integrate automatic sync with taskwarrior cli. Would it launch a background task like in the example? Is that something we can do reliably in a cross-platform compatible way? Is that the behavior users would expect and want?

I do agree that launching background processes does make it a bit hairy. I'm not toooo experienced with these things, but here's my two cents:

I was thinking that, honestly, syncing is fast enough in most cases where I don't think it's entirely unreasonable to just block on sync. That simplifies the process. If it takes longer than, say, 100ms we could show a message and the user could just ctrl-c it. However, if some client uses the cli (like taskwarrior-tui or the android app) then you don't have that control, and honestly it does feel that in those cases the client should be the one handling the syncing, not the cli... Also, it seems excessive to sync after every command, and some debouncing would be nice. That's harder to do in a cross-platform way, but in linux/macos it should be easy enough (with fork, iiuc) and in windows I would assume there is a way to do it too (apparently, CreateProcess).

Also, blocking before displaying a task seems terrible, so maybe the syncing should only be automatically done on modify? Perhaps the cron job is good enough for "pulling" changes, because generally I care more about having all my changes pushed all the time so I can access them if I don't have access to the primary machine. But it would be nice to have it in the documentation as recommended and some pointers on how to do it :)

---

Generally, I liked the idea of having some auto-sync built in taskwarrior because then it can be consistent behavior across clients (for example, I can't set up hooks on the android app so I have to manually sync there). I think it's also nice for people that come from apps like Todoist, TickTick, Microsoft To-Do, etc., because that's generally how it works there.

[djmitche]

An issue with auto-debouncing is, you don't know when the next invocation of task will occur. So for example if I quickly create three tasks task add bake bread task add prepare TPS report task add revise resignation letter, then it would be great to only sync once after the third command. But how do we know there won't be a fourth?

[djmitche]

BTW, here's the heredoc I use to create a launchd plist on macOS:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.example.${name}</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/bash</string>
        <string>-c</string>
        <string>${command}</string>
    </array>
    <key>StartInterval</key>
    <integer>${interval}</integer>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/periodic-${name}.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/periodic-${name}.log</string>
</dict>
</plist>

[odilf]

An issue with auto-debouncing is, you don't know when the next invocation of task will occur. [...]

What I do in the script is, on modify, queue all the changes and sync no more than every 5 (or n) seconds. So batch all the changes done in n second intervals, mostly to avoid spamming the serve. For "pulling", I check on launch if I have done a sync in the past 5 minutes and do it if I haven't. I think that's reasonable, maybe with different delays too.

In fact, the denouncing of modifying is tricky to do using hooks, because the hook waits for the process to be done, even if you do cmd & or disown, so unless you use nohup, every modification takes 5 seconds to do (and I'm still not entirely sure if nohup is the "correct" option here 😅), which is another reason why it would be nice to have it built-in.

BTW, here's the heredoc I use to create a launchd plist on macOS:

oh, thanks 😊

[ryneeverett]

I guess another option emerging is to add a document of "recipes" for automatic sync with various service managers. I thought there used to be such a document but it might have gotten wiped out during the taskchampion transition.