Commit Graph

306 Commits

Author SHA1 Message Date
Damien Elmes
5f0a74e01f Show tooltip when auto-advance enabled/disabled 2023-12-11 14:43:12 +10:00
Damien Elmes
c09a46f857 Rework rollover check
- Avoid waking up frequently
- Ensure collection access happens before sync begins, as otherwise
it may block the UI
2023-12-10 12:37:01 +10:00
Abdo
843c0417b0
Fix auto advance state not being restored after focus change in some (#2860)
cases
2023-12-04 14:54:25 +10:00
Abdo
ae7b14bf40
Add auto-advance options to deck preset (#2765)
* Move stop-timer-on-answer strings to correct section

* Add auto-advance options to deck preset

* Implement answer actions

* Fix error when last card is answered before timeout

* Fix deserialization of answerAction

* Add answerAction to reserved key list

* Fix inverted boolean

* Add option to wait for audio to finish

* Add auto-advance toggle

* Add shortcut

* Disable auto-advance when main window state changes

* Start auto-advance timer after option is toggled

* Disable auto-advance when main window loses focus

* Use existing translations

* Add Answer Hard and Show Reminder
2023-11-13 10:41:51 +10:00
Damien Elmes
ab3a37ff71 Fix 'creating backup' showing for longer than necessary 2023-11-03 22:18:30 +10:00
Damien Elmes
865296254a Tweaks to add-on startup failure screen / update checks
- Add a Check for Updates button to the screen
- Make the update list screen non-modal, so that other modal pop-ups
at startup don't leave the user stuck
- When manually checking for updates, update Anki's last check time
2023-10-26 11:23:24 +10:00
Abdo
0ce9b3f0a1
Refresh reviewer as the day rolls over (#2756) 2023-10-21 12:18:01 +10:00
Abdo
b23f17df27
Drop support for checkpoints (#2742)
* Drop support for checkpoints

* Deprecate .flush()

* Remove .begin/.commit

* Remove rollback() and deprecate save/autosave/reset()

There's no need to commit anymore, as the Rust code is handling
transactions for us.

* Add safer transact() method

This will ensure add-on authors can't accidentally leave a transaction
open, leading to data loss.

---------

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2023-10-17 12:43:34 +10:00
chandraiyengar
b35a11ffd6
Add profile name in profile deletion confirmation message (#2722)
* Add profile name in profile deletion confirmation message

* Update contributors

* leave old translation and add new one
2023-10-12 13:34:52 +10:00
Abdo
8b0d663fd0
Allow full screen mode on new Qt6 drivers (#2660)
* Allow full screen mode on new Qt6 drivers

* Update fulscreen warning message
2023-09-16 11:28:20 +10:00
Damien Elmes
bfef908c6c
Refactor media sync handling (#2647)
* Refactor media sync handling

- The media USN is now returned in sync/meta, which avoids an extra
round-trip.
- Media syncing is now automatically started by the syncing code at
the end of a normal or full sync, which avoids it competing for bandwidth
and resources, and avoids duplicate invalid login messages when the auth
token is invalid.
- Added a new media_sync_progress() method to both check if media is
syncing, and get access to the latest progress.
- Updated the sync log screen to only show the latest line, like AnkiMobile.
- Show media sync errors in a pop-up, so they don't get missed. Use a non-modal
pop-up to avoid potential conflicts with other modals.

* Remove print statement
2023-09-10 13:22:20 +10:00
Damien Elmes
a3d38bdad3 Fix syncStatus being called twice when auto sync disabled 2023-09-07 15:11:36 +10:00
Damien Elmes
ffd392de21
Change Anki's version scheme; bump to 23.09 (#2640)
* Accept iterables as inputs to backend methods

* Shift add-on check to backend; use new endpoint

The new endpoint will return info on a suitable branch if found,
instead of returning all branches. This simplifies the frontend code,
and means that you can now drop support for certain versions without
it also remotely disabling the add-on for people who are running one of
the excluded versions, like in

https://forums.ankiweb.net/t/prevent-add-ons-from-being-disabled-remote-stealthily-surreptitiously/33427

* Bump version to 23.09

This changes Anki's version numbering system to year.month.patch, as
previously mentioned on https://forums.ankiweb.net/t/use-a-different-versioning-system-semver-perhaps/20046/5

This is shaping up to be a big release, with the introduction of FSRS and
image occlusion, and it seems like a good time to be finally updating the
version scheme as well. AnkiWeb has been updated to understand the new
format, and add-on authors will now specify version compatibility using
the full version number, as can be seen here:

https://ankiweb.net/shared/info/3918629684

* Shift update check to backend, and tidy up update.py

* Use the shared client for sync connections too
2023-09-07 12:37:15 +10:00
Damien Elmes
4691560522 Work around CI failure on Windows with PyQt 6.5.2 2023-07-26 21:10:19 +10:00
Luc Mcgrady
bceffd96c6
Add keyboard shortcut for legacy stats (#2583)
* Added: Shortcut for legacy stats

* Added: Name to contributors

* Capitalize S
2023-07-24 11:31:41 +10:00
RumovZ
9430e3ddf2
Fix addon update dialog showing twice (#2580)
* Gitignore .ninja_deps

* Don't check for addon updates on profile unload

This fixes two update dialogs showing up when switching profiles.

* Only show addon update dialog once

... instead of once every time the profile is changed, when the version
doesn't match.
This is analogous to when the dialog is triggered by elapsed time.

* Manage last_run_version update in ProfileManager
2023-07-18 22:26:59 +10:00
Abdo
0100ae7431
Fix main window geometry/state not being restored in some cases (#2558)
Introduced by 19b08eb280

Report: https://forums.ankiweb.net/t/65-the-size-of-the-main-window-is-reset-every-time-it-is-closed-again/31364
2023-06-27 16:52:34 +10:00
Ren Tatsumoto
9c5edfa88c
Ensure there's no duplicate shortcuts after running state_shortcuts_will_change (#2509)
* remove duplicate shortucts after running hook

* normalize shortcuts by converting them to QKeySequence

* no need to copy here

* extract method
2023-05-22 14:07:15 +10:00
Damien Elmes
e6f970e969 Tolerate missing keys in profile DB
https://forums.ankiweb.net/t/crash-with-error-keyerror-mainwindowstate/29689
2023-04-26 19:07:59 +10:00
RumovZ
b55161cd39
Improve debug console (#2435)
* List actions and locals in debug console

* Ignore whitespace when wrapping line with pp

* Scroll down after printing in debug console

Was previously preserving relative vertical position.

* Add feature to open and save debug scripts

* Refactor debug console into own module

* Add buffers to switch scripts

* Add action to delete script
2023-03-15 15:29:05 +10:00
Damien Elmes
d1abfbc6a1 Passing an invalid profile name on the commandline will now show picker
https://forums.ankiweb.net/t/open-with-profiles-to-start/27612
2023-03-02 17:13:04 +10:00
evandrocoan
d9f1e22648
Create the hook will_show_web to control html5 media elements with Javascript (#2340)
* Replaced ankimedia object directly call by addon specific hook

# Conflicts:
#	qt/aqt/browser/previewer.py
#	qt/aqt/clayout.py
#	qt/aqt/reviewer.py

* Replaced ankimedia.js by addon specific hook

# Conflicts:
#	qt/aqt/browser/previewer.py
#	qt/aqt/clayout.py
#	qt/aqt/main.py

* Create specific location name for each hook to reuse control

* Created the card_review_webview_did_init hook

* Extended the hook card_will_show to replace will_show_web

The new hook card_will_show_state takes three new arguments

* Created the hook audio_did_pause_or_unpause to replace will_show_web

The new hook is called when audio toggle pause is called

* Created the hook audio_will_replay to replace will_show_web

The new hook is called when the audio is replayed by the user.

* Created the hook previewer_will_redraw_after_show_both_sides_toggled

to replace will_show_web.
The new hook fully replaces the last uses of will_show_web.

* Replaced card_will_show_state hook with reviewer_did_init and

equivalents. Instead of receiving the required state, it access it
by caching the object values with hooks as reviewer_did_init.
2023-02-13 14:50:26 +10:00
Aristotelis
0f86c9fd11
Rework & unify webview identification and title setting (#2366)
* Create common web view registry and unify title setting

* Consistently use space-separated naming for webview titles

None of the modified titles seem to be in use by add-ons, so we are not bound to the current naming.

The old naming was likely following camelCase as the name was also acting as a key for saveGeom, which is no longer the case.

* Update webview_did_inject_style_into_page example

* Add docstring to addon-targeted method

* Change AnkiWebView.origin to property

* Fix dupe enum value

* Tweak method name

* Add semicolon

* Rename `AnkiWebViewOrigin` to `AnkiWebViewKind`
2023-02-10 14:53:11 +10:00
evandrocoan
13d557589f
Improve info message (#2343) 2023-01-29 12:55:01 +10:00
Damien Elmes
4c98a1a318 Fix error shown after toggling full screen
https://forums.ankiweb.net/t/anki-2-1-57-beta/26494/62
2023-01-29 12:41:28 +10:00
Aristotelis
476e3af759
Bump Linux theme refresh timer back up to 5 minutes (#2331)
On Linux distributions that do not yet support org.freedesktop.appearance.color-scheme (most distros released prior to 2022), querying DBus for the missing property produces verbose journal entries over Anki's entire runtime.
2023-01-18 22:24:59 +10:00
Matthias Metelka
f169ee0933
Revamp Preferences, implement Minimalist Mode and Qt widget gallery to test GUI changes (#2289)
* Create widget gallery dialog

* Add WidgetGallery to debug dialog

* Use enum for its intended purpose

* Rename "reduced-motion" to "reduce-motion"

* Add another border-radius value

and make former large radius a bit smaller.

* Revamp preferences, add minimalist mode

Also:
- create additional and missing widget styles and tweak existing ones
- use single profile entry to set widget styles and reduce choices to Anki and Native

* Indent QTabBar style definitions

* Add missing styles for QPushButton states

* Fix QTableView background

* Remove unused layout from Preferences

* Fix QTabView focused tab style

* Highlight QCheckBox and QRadioButton when focused

* Fix toolbar styles

* Reorder preferences

* Add setting to hide bottom toolbar

* Move toolbar settings above minimalist modes

* Remove unused lines

* Implement proper full-screen mode

* Sort imports

* Tweak deck overview appearance in minimalist mode

* Undo TitledContainer changes

since nobody asked for that

* Remove dynamic toolbar background from minimalist mode

* Tweak buttons in minimalist mode

* Fix some issues

* Reduce theme check interval to 5s on Linux

* Increase hide timer interval to 2s

* Collapse toolbars with slight delay when moving to review state

This should ensure the bottom toolbar collapses too.

* Allow users to make hiding exclusive to full screen

* Rename full screen option

* Fix hide mode dropdown ignoring checkbox state on startup

* Fix typing issue

* Refine background image handling

Giving the toolbar body the main webview height ensures background-size: cover behaves exactly the same.

To prevent an override of other background properties, users are advised to only set background-images via the background-image property, not the background shorthand.

* Fix top toolbar getting huge when switching modes

The issue was caused by the min-height hack to align the background images. A call to web.adjustHeightToFit would set the toolbar to the same height as the main webview, as the function makes use of document.offsetHeight.

* Prevent scrollbar from appearing on bottom toolbar resize

* Cleanup

* Put review tab before editing; fix some tab orders

* Rename 'network' to 'syncing'

* Fix bottom toolbar disappearing on UI > 100

* Improve Preferences layout by adding vertical spacers to the bottom

also make the hiding of video_driver and its label more obvious in preferences.py.

* Fix bottom toolbar animating on startup

Also fix bottom toolbar not appearing when unchecking hide mode in reviewer.

* Hide/Show menubar in fullscreen mode along with toolbar

* Attempt to fix broken native theme on macOS

* Format

* Improve native theme on other systems by not forcing palette

with the caveat that theme switching can get weird.

* Fix theme switching in native style

* Remove redundant condition

* Add back check for Qt5 to prevent theme issues

* Add check for macOS before setting fusion theme

* Do not force scrollbar styles on macOS

* Remove all of that crazy theme logic

* Use canvas instead of button-bg for ColorRole.Button

* Make sure Anki style is always based on Fusion

otherwise we can't guarantee the same look on all systems.

* Explicitly apply default style when Anki style is not selected

This should fix the style not switching back after it was selected.

* Remove reduncant default_palette

* Revert 8af4c1cc2

On Mac with native theme, both Qt5 and Qt6 look correct already. On
the Anki theme, without this change, we get the fusion-style scrollbars
instead of the rounded ones.

* Rename AnkiStyles enum to WidgetStyle

* Fix theme switching shades on same theme

* Format

* Remove unused placeholderText

that caused an error when opening the widget gallery on Qt5.

* Check for full screen windowState using bitwise operator

to prevent error in Qt5.

Credit: https://stackoverflow.com/a/65425151

* Hide style option on Windows

also exclude native option from dropdown just in case.

* Format

* Minor naming tweak
2023-01-18 21:24:16 +10:00
Matthias Metelka
9f8667fb47
Auto-hide toolbar in Reviewer (#2262)
* Give webviews a slide-in animation

if reduced motion isn't set.

* Auto-hide toolbar in review mode

moving the mouse above the main webview expands the toolbar. When the mouse leaves the toolbar, it will collapse after a delay of 2s.

* Save some space on bottom toolbars

* Use props for all hard-coded transition durations

and decrease most commonly used duration (200ms) to 150ms.

* Move auto-hide logic into ToolbarWebView

and handle auto-hide specific events in the respective webview subclasses.

* Fix typing issues

* Fix flickering issue

* Add auto_hide_toolbar opt-in to preferences

* Rename hide_toolbar to collapse_toolbar

to better describe the dock-like behaviour.

* Rename setting to minimize_distractions

* Reduce calls to pm in eventFilter

* Run formatter

* Revert setting title to something more specific

* Increase default animation time to 180ms

* Inset toolbar in review mode

when auto-hide is not enabled.

* Use card background on toolbar and add glass effect

* Use flatten/elevate over inset/outset

* Use flatten/elevate over inset/outset

* Update toolbar.py

* Fix toolbar background delay

* Tweak styles

* Use "collapse" instead of "auto-hide"

* Fix background misalignment in collapse mode

* Do not collapse toolbar when pointer is outside MainWebView

* Reduce hide_timer interval to 1000ms

* Use CSS to hide toolbar instead of setting webview height

* Add guard to prevent backdrop-filter: blur on Qt 5.14

* Apply transition to body instead of toolbar

to not complicate things for #2301.

* Fix Qt 5.14 and apply guard globally

* Fix background image scaling difference

* Tweak preference wording (dae)
2023-01-09 14:39:31 +10:00
Damien Elmes
ba68764fcb Another attempt at fixing missing cacert.pem
A few reports like https://forums.ankiweb.net/t/error-report-check-database-did-not-work/25796
indicate that the previous solution has not helped, and has just made things
worse. My guess is that AppNap on macOS is preventing the timer from even
running before the file gets cleaned up.
2022-12-30 15:30:53 +10:00
Damien Elmes
9e8deb6287 Handle case where Linux distros patch out certifi library
19deb7ad25 (commitcomment-93512291)
2022-12-20 11:21:09 +10:00
Damien Elmes
f2ca87b1d8 Ensure certifi refresh repeats 2022-12-03 23:31:08 +10:00
Damien Elmes
d98912ca6b Revert "Probable workaround for mpv failing after a few days on macOS"
This reverts commit fa4fc3e15a.

Issue turned out to be a packaging problem, and this should not be
required as the socket should be held open even if removed.
2022-12-03 23:30:19 +10:00
Damien Elmes
fa4fc3e15a Probable workaround for mpv failing after a few days on macOS
Also fix the certifi bumper: it was supposed to repeat
2022-12-03 22:11:39 +10:00
Damien Elmes
509e0a9811 Try to address crashes on shutdown
I was able to reproduce the crashes fairly reliably by opening the
prefs screen on startup and shutting down the app after 600ms; after
this change the crashes no longer seem to occur.
2022-12-03 19:33:51 +10:00
Damien Elmes
5e0a761b87
Move away from Bazel (#2202)
(for upgrading users, please see the notes at the bottom)

Bazel brought a lot of nice things to the table, such as rebuilds based on
content changes instead of modification times, caching of build products,
detection of incorrect build rules via a sandbox, and so on. Rewriting the build
in Bazel was also an opportunity to improve on the Makefile-based build we had
prior, which was pretty poor: most dependencies were external or not pinned, and
the build graph was poorly defined and mostly serialized. It was not uncommon
for fresh checkouts to fail due to floating dependencies, or for things to break
when trying to switch to an older commit.

For day-to-day development, I think Bazel served us reasonably well - we could
generally switch between branches while being confident that builds would be
correct and reasonably fast, and not require full rebuilds (except on Windows,
where the lack of a sandbox and the TS rules would cause build breakages when TS
files were renamed/removed).

Bazel achieves that reliability by defining rules for each programming language
that define how source files should be turned into outputs. For the rules to
work with Bazel's sandboxing approach, they often have to reimplement or
partially bypass the standard tools that each programming language provides. The
Rust rules call Rust's compiler directly for example, instead of using Cargo,
and the Python rules extract each PyPi package into a separate folder that gets
added to sys.path.

These separate language rules allow proper declaration of inputs and outputs,
and offer some advantages such as caching of build products and fine-grained
dependency installation. But they also bring some downsides:

- The rules don't always support use-cases/platforms that the standard language
tools do, meaning they need to be patched to be used. I've had to contribute a
number of patches to the Rust, Python and JS rules to unblock various issues.
- The dependencies we use with each language sometimes make assumptions that do
not hold in Bazel, meaning they either need to be pinned or patched, or the
language rules need to be adjusted to accommodate them.

I was hopeful that after the initial setup work, things would be relatively
smooth-sailing. Unfortunately, that has not proved to be the case. Things
frequently broke when dependencies or the language rules were updated, and I
began to get frustrated at the amount of Anki development time I was instead
spending on build system upkeep. It's now about 2 years since switching to
Bazel, and I think it's time to cut losses, and switch to something else that's
a better fit.

The new build system is based on a small build tool called Ninja, and some
custom Rust code in build/. This means that to build Anki, Bazel is no longer
required, but Ninja and Rust need to be installed on your system. Python and
Node toolchains are automatically downloaded like in Bazel.

This new build system should result in faster builds in some cases:

- Because we're using cargo to build now, Rust builds are able to take advantage
of pipelining and incremental debug builds, which we didn't have with Bazel.
It's also easier to override the default linker on Linux/macOS, which can
further improve speeds.
- External Rust crates are now built with opt=1, which improves performance
of debug builds.
- Esbuild is now used to transpile TypeScript, instead of invoking the TypeScript
compiler. This results in faster builds, by deferring typechecking to test/check
time, and by allowing more work to happen in parallel.

As an example of the differences, when testing with the mold linker on Linux,
adding a new message to tags.proto (which triggers a recompile of the bulk of
the Rust and TypeScript code) results in a compile that goes from about 22s on
Bazel to about 7s in the new system. With the standard linker, it's about 9s.

Some other changes of note:

- Our Rust workspace now uses cargo-hakari to ensure all packages agree on
available features, preventing unnecessary rebuilds.
- pylib/anki is now a PEP420 implicit namespace, avoiding the need to merge
source files and generated files into a single folder for running. By telling
VSCode about the extra search path, code completion now works with generated
files without needing to symlink them into the source folder.
- qt/aqt can't use PEP420 as it's difficult to get rid of aqt/__init__.py.
Instead, the generated files are now placed in a separate _aqt package that's
added to the path.
- ts/lib is now exposed as @tslib, so the source code and generated code can be
provided under the same namespace without a merging step.
- MyPy and PyLint are now invoked once for the entire codebase.
- dprint will be used to format TypeScript/json files in the future instead of
the slower prettier (currently turned off to avoid causing conflicts). It can
automatically defer to prettier when formatting Svelte files.
- svelte-check is now used for typechecking our Svelte code, which revealed a
few typing issues that went undetected with the old system.
- The Jest unit tests now work on Windows as well.

If you're upgrading from Bazel, updated usage instructions are in docs/development.md and docs/build.md. A summary of the changes:

- please remove node_modules and .bazel
- install rustup (https://rustup.rs/)
- install rsync if not already installed  (on windows, use pacman - see docs/windows.md)
- install Ninja (unzip from https://github.com/ninja-build/ninja/releases/tag/v1.11.1 and
  place on your path, or from your distro/homebrew if it's 1.10+)
- update .vscode/settings.json from .vscode.dist
2022-11-27 15:24:20 +10:00
Damien Elmes
e5c4ccf08c If a backup fails before close, ensure collection closed
Otherwise when user returns to profiles screen, they'll be unable to
open a different profile, as the collection is still open.

Encountered when opening the collection that triggered
https://github.com/ankitects/anki/issues/2123
2022-10-29 11:08:58 +10:00
Abdo
497b246b66
Add MainWindowState hints to more signatures (#2064) 2022-09-12 19:31:22 +10:00
Damien Elmes
19deb7ad25 Access certificates periodically
Tentatively closes #1965
2022-08-19 10:00:38 +10:00
Sam Penny
21812556a6
Serialize modal popups at startup (#2000)
* call_add-on_update_after_initial_sync

* Add safeMode restriction

* check for anki update after other modal dialogs

* fix failing test

* make changes suggested in review
2022-08-09 13:40:52 +10:00
Damien Elmes
d482e90c6b Turn new import/export code on by default
Closes #1966
2022-07-18 13:31:24 +10:00
Sam Penny
662586765b
Load previously loaded profile (#1960)
* added option to load the last loaded profile

* add formatting changes for tests

* fix comments - integrate as new functionality rather than as an option, fix type hinting and remove bug if profile is deleted
2022-07-18 13:01:36 +10:00
Mateus Etto
af8ae69837
Fix Study Deck in Sway spawning tiled window instead of floating window (#1935)
* Fix Study Deck in sway spawning tiled window instead of floating window

* Update CONTRIBUTORS

* Fix format error
2022-06-27 17:27:56 +10:00
Abdo
67f64740c4
Accept .zip as an alias for .ankiaddon too (#1914) 2022-06-17 11:10:29 +10:00
Damien Elmes
940f1dea06 Clear cached flag names on sync
Closes #1907
2022-06-09 12:08:16 +10:00
Damien Elmes
82196753ec Rework display of available cards in custom study
In v3, it's more informative to show the count of child decks separately,
since increasing the limit of the current deck does not increase the limits
of child decks. When we rework the decks list in the future, a tooltip
will hopefully provide an easier way for users to see where cards are
available, and where limits are being applied.

Closes #1868
2022-05-20 17:52:02 +10:00
Damien Elmes
fdbe211539 Support new import code in drag/drop + file double-click cases
https://forums.ankiweb.net/t/anki-2-1-52-release-candidate/19412/24
2022-05-03 14:09:40 +10:00
RumovZ
5f9451f547
Add apkg import/export on backend (#1743)
* Add apkg export on backend

* Filter out missing media-paths at write time

* Make TagMatcher::new() infallible

* Gather export data instead of copying directly

* Revert changes to rslib/src/tags/

* Reuse filename_is_safe/check_filename_safe()

* Accept func to produce MediaIter in export_apkg()

* Only store file folder once in MediaIter

* Use temporary tables for gathering

export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.

* Use schedule_as_new() to reset cards

* ExportData → ExchangeData

* Ignore ascii case when filtering system tags

* search_notes_cards_into_table →

search_cards_of_notes_into_table

* Start on apkg importing on backend

* Fix due dates in days for apkg export

* Refactor import-export/package

- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.

* Add SafeMediaEntry for deserialized MediaEntries

* Prepare media based on checksums

- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.

* Handle encoding in `replace_media_refs()`

* Add trait to keep down cow boilerplate

* Add notetypes immediately instaed of preparing

* Move target_col into Context

* Add notes immediately instaed of preparing

* Note id, not guid of conflicting notes

* Add import_decks()

* decks_configs → deck_configs

* Add import_deck_configs()

* Add import_cards(), import_revlog()

* Use dyn instead of generic for media_fn

Otherwise, would have to pass None with type annotation in the default
case.

* Fix signature of import_apkg()

* Fix search_cards_of_notes_into_table()

* Test new functions in text.rs

* Add roundtrip test for apkg (stub)

* Keep source id of imported cards (or skip)

* Keep source ids of imported revlog (or skip)

* Try to keep source ids of imported notes

* Make adding notetype with id undoable

* Wrap apkg import in transaction

* Keep source ids of imported deck configs (or skip)

* Handle card due dates and original due/did

* Fix importing cards/revlog

Card ids are manually uniquified.

* Factor out card importing

* Refactor card and revlog importing

* Factor out card importing

Also handle missing parents .

* Factor out note importing

* Factor out media importing

* Maybe upgrade scheduler of apkg

* Fix parent deck gathering

* Unconditionally import static media

* Fix deck importing edge cases

Test those edge cases, and add some global test helpers.

* Test note importing

* Let import_apkg() take a progress func

* Expand roundtrip apkg test

* Use fat pointer to avoid propogating generics

* Fix progress_fn type

* Expose apkg export/import on backend

* Return note log when importing apkg

* Fix archived collection name on apkg import

* Add CollectionOpWithBackendProgress

* Fix wrong Interrupted Exception being checked

* Add ClosedCollectionOp

* Add note ids to log and strip HTML

* Update progress when checking incoming media too

* Conditionally enable new importing in GUI

* Fix all_checksums() for media import

Entries of deleted files are nulled, not removed.

* Make apkg exporting on backend abortable

* Return number of notes imported from apkg

* Fix exception printing for QueryOp as well

* Add QueryOpWithBackendProgress

Also support backend exporting progress.

* Expose new apkg and colpkg exporting

* Open transaction in insert_data()

Was slowing down exporting by several orders of magnitude.

* Handle zstd-compressed apkg

* Add legacy arg to ExportAnkiPackage

Currently not exposed on the frontend

* Remove unused import in proto file

* Add symlink for typechecking of import_export_pb2

* Avoid kwargs in pb message creation, so typechecking is not lost

Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign 
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.

* Avoid re-exporting protobuf msgs we only use internally

* Stop after one test failure

mypy often fails much faster than pylint

* Avoid an extra allocation when extracting media checksums

* Update progress after prepare_media() finishes

Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.

* Show progress of note imports

Note import is the slowest part, so showing progress here makes the UI
feel more responsive.

* Reset filtered decks at import time

Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.

We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).

https://github.com/ankitects/anki/pull/1743/files#r839346423

* Fix a corner-case where due dates were shifted by a day

This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.

* Log conflicting note in remapped nt case

* take_fields() → into_fields()

* Alias `[u8; 20]` with `Sha1Hash`

* Truncate logged fields

* Rework apkg note import tests

- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)

* Fix sorting of imported decks

Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.

* target[_id]s → existing_card[_id]s

* export_collection_extracting_media() → ...

export_into_collection_file()

* target_already_exists→card_ordinal_already_exists

* Add search_cards_of_notes_into_table.sql

* Imrove type of apkg export selector/limit

* Remove redundant call to mod_schema()

* Parent tooltips to mw

* Fix a crash when truncating note text

String::truncate() is a bit of a footgun, and I've hit this before
too :-)

* Remove ExportLimit in favour of separate classes

* Remove OpWithBackendProgress and ClosedCollectionOp

Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.

Also fix aborting of colpkg exports, which slipped through in #1817.

* Tidy up import log

* Avoid QDialog.exec()

* Default to excluding scheuling for deck list deck

* Use IncrementalProgress in whole import_export code

* Compare checksums when importing colpkgs

* Avoid registering changes if hashes are not needed

* ImportProgress::Collection → ImportProgress::File

* Make downgrading apkgs depend on meta version

* Generalise IncrementableProgress

And use it in entire import_export code instead.

* Fix type complexity lint

* Take count_map for IncrementableProgress::get_inner

* Replace import/export env with Shift click

* Accept all args from update() for backend progress

* Pass fields of ProgressUpdate explicitly

* Move update_interval into IncrementableProgress

* Outsource incrementing into Incrementor

* Mutate ProgressUpdate in progress_update callback

* Switch import/export legacy toggle to profile setting

Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.

One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.

* Show extension in export dialog

* Continue to provide separate options for schema 11+18 colpkg export

* Default to colpkg export when using File>Export

* Improve appearance of combo boxes when switching between apkg/colpkg

+ Deal with long deck names

* Convert newlines to spaces when showing fields from import

Ensures each imported note appears on a separate line

* Don't separate total note count from the other summary lines

This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.

* Fix 'deck not normal' error when importing a filtered deck for the 2nd time

* Fix [Identical] being shown on first import

* Revert "Continue to provide separate options for schema 11+18 colpkg export"

This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.

Will use a different approach

* Move legacy support into a separate exporter option; add to apkg export

* Adjust 'too new' message to also apply to .apkg import case

* Show a better message when attempting to import new apkg into old code

Previously the user could end seeing a message like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

Unfortunately we can't retroactively fix this for older clients.

* Hide legacy support option in older exporting screen

* Reflect change from paths to fnames in type & name

* Make imported decks normal at once

Then skip special casing in update_deck(). Also skip updating
description if new one is empty.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-05-02 21:12:46 +10:00
Damien Elmes
a7cb5e210e Close browser before full sync or colpkg import/export
Closes #1797
2022-04-19 17:10:34 +10:00
Aristotelis
8100e81789
Fix a number of bugs with add-on conflict resolution (#1780)
* Always enable manually installed add-ons

Ensures that manually installed add-ons are enabled after the installation, even if previously disabled.

Prevents scenarios where users could end up with no active add-on build (e.g. when switching between stable add-on builds distributed via AnkiWeb and betas distributed via GitHub).

* Improve type annotations

* Also enable disabled AnkiWeb add-ons upon interactive installation

Applies to add-ons that users actively install via their AnkiWeb ID. Updates are exempt, preserving whatever status add-ons were in.

* Prevent disabled add-ons from triggering conflicts

* Fix download_addons() not passing on force_enable argument (dae)
2022-04-09 13:51:59 +10:00
Damien Elmes
4515c41d2c
Backup improvements (#1728)
* Collection needs to be closed prior to backup even when not downgrading

* Backups -> BackupLimits

* Some improvements to backup_task

- backup_inner now returns the error instead of logging it, so that
the frontend can discover the issue when they await a backup (or create
another one)
- start_backup() was acquiring backup_task twice, and if another thread
started a backup between the two locks, the task could have been accidentally
overwritten without awaiting it

* Backups no longer require a collection close

- Instead of closing the collection, we ensure there is no active
transaction, and flush the WAL to disk. This means the undo history
is no longer lost on backup, which will be particularly useful if we
add a periodic backup in the future.
- Because a close is no longer required, backups are now achieved with
a separate command, instead of being included in CloseCollection().
- Full sync no longer requires an extra close+reopen step, and we now
wait for the backup to complete before proceeding.
- Create a backup before 'check db'

* Add File>Create Backup

https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157

* Defer checkpoint until we know we need it

When running periodic backups on a timer, we don't want to be fsync()ing
unnecessarily.

* Skip backup if modification time has not changed

We don't want the user leaving Anki open overnight, and coming back
to lots of identical backups.

* Periodic backups

Creates an automatic backup every 30 minutes if the collection has been
modified.

If there's a legacy checkpoint active, tries again 5 minutes later.

* Switch to a user-configurable backup duration

CreateBackup() now uses a simple force argument to determine whether
the user's limits should be respected or not, and only potentially
destructive ops (full download, check DB) override the user's configured
limit.

I considered having a separate limit for collection close and automatic
backups (eg keeping the previous 5 minute limit for collection close),
but that had two downsides:

- When the user closes their collection at the end of the day, they'd
get a recent backup. When they open the collection the next day, it
would get backed up again within 5 minutes, even though not much had
changed.
- Multiple limits are harder to communicate to users in the UI

Some remaining decisions I wasn't 100% sure about:

- If force is true but the collection has not been modified, the backup
will be skipped. If the user manually deleted their backups without
closing Anki, they wouldn't get a new one if the mtime hadn't changed.
- Force takes preference over the configured backup interval - should
we be ignored the user here, or take no backups at all?

Did a sneaky edit of the existing ftl string, as it hasn't been live
long.

* Move maybe_backup() into Collection

* Use a single method for manual and periodic backups

When manually creating a backup via the File menu, we no longer make
the user wait until the backup completes. As we continue waiting for
the backup in the background, if any errors occur, the user will get
notified about it fairly quickly.

* Show message to user if backup was skipped due to no changes

+ Don't incorrectly assert a backup will be created on force

* Add "automatic" to description

* Ensure we backup prior to importing colpkg if collection open

The backup doesn't happen when invoked from 'open backup' in the profile
screen, which matches Anki's previous behaviour. The user could
potentially clobber up to 30 minutes of their work if they exited to
the profile screen and restored a backup, but the alternative is we
create backups every time a backup is restored, which may happen a number
of times if the user is trying various ones. Or we could go back to a
separate throttle amount for this case, at the cost of more complexity.

* Remove the 0 special case on backup interval; minimum of 5 minutes

https://github.com/ankitects/anki/pull/1728#discussion_r830876833
2022-03-21 19:40:42 +10:00