Commit Graph

117 Commits

Author SHA1 Message Date
Matthias Metelka
10149f7cb0
Change names of edited translations to notify translators (#2338) 2023-01-19 14:57:55 +10:00
Matthias Metelka
f169ee0933
Revamp Preferences, implement Minimalist Mode and Qt widget gallery to test GUI changes (#2289)
* Create widget gallery dialog

* Add WidgetGallery to debug dialog

* Use enum for its intended purpose

* Rename "reduced-motion" to "reduce-motion"

* Add another border-radius value

and make former large radius a bit smaller.

* Revamp preferences, add minimalist mode

Also:
- create additional and missing widget styles and tweak existing ones
- use single profile entry to set widget styles and reduce choices to Anki and Native

* Indent QTabBar style definitions

* Add missing styles for QPushButton states

* Fix QTableView background

* Remove unused layout from Preferences

* Fix QTabView focused tab style

* Highlight QCheckBox and QRadioButton when focused

* Fix toolbar styles

* Reorder preferences

* Add setting to hide bottom toolbar

* Move toolbar settings above minimalist modes

* Remove unused lines

* Implement proper full-screen mode

* Sort imports

* Tweak deck overview appearance in minimalist mode

* Undo TitledContainer changes

since nobody asked for that

* Remove dynamic toolbar background from minimalist mode

* Tweak buttons in minimalist mode

* Fix some issues

* Reduce theme check interval to 5s on Linux

* Increase hide timer interval to 2s

* Collapse toolbars with slight delay when moving to review state

This should ensure the bottom toolbar collapses too.

* Allow users to make hiding exclusive to full screen

* Rename full screen option

* Fix hide mode dropdown ignoring checkbox state on startup

* Fix typing issue

* Refine background image handling

Giving the toolbar body the main webview height ensures background-size: cover behaves exactly the same.

To prevent an override of other background properties, users are advised to only set background-images via the background-image property, not the background shorthand.

* Fix top toolbar getting huge when switching modes

The issue was caused by the min-height hack to align the background images. A call to web.adjustHeightToFit would set the toolbar to the same height as the main webview, as the function makes use of document.offsetHeight.

* Prevent scrollbar from appearing on bottom toolbar resize

* Cleanup

* Put review tab before editing; fix some tab orders

* Rename 'network' to 'syncing'

* Fix bottom toolbar disappearing on UI > 100

* Improve Preferences layout by adding vertical spacers to the bottom

also make the hiding of video_driver and its label more obvious in preferences.py.

* Fix bottom toolbar animating on startup

Also fix bottom toolbar not appearing when unchecking hide mode in reviewer.

* Hide/Show menubar in fullscreen mode along with toolbar

* Attempt to fix broken native theme on macOS

* Format

* Improve native theme on other systems by not forcing palette

with the caveat that theme switching can get weird.

* Fix theme switching in native style

* Remove redundant condition

* Add back check for Qt5 to prevent theme issues

* Add check for macOS before setting fusion theme

* Do not force scrollbar styles on macOS

* Remove all of that crazy theme logic

* Use canvas instead of button-bg for ColorRole.Button

* Make sure Anki style is always based on Fusion

otherwise we can't guarantee the same look on all systems.

* Explicitly apply default style when Anki style is not selected

This should fix the style not switching back after it was selected.

* Remove reduncant default_palette

* Revert 8af4c1cc2

On Mac with native theme, both Qt5 and Qt6 look correct already. On
the Anki theme, without this change, we get the fusion-style scrollbars
instead of the rounded ones.

* Rename AnkiStyles enum to WidgetStyle

* Fix theme switching shades on same theme

* Format

* Remove unused placeholderText

that caused an error when opening the widget gallery on Qt5.

* Check for full screen windowState using bitwise operator

to prevent error in Qt5.

Credit: https://stackoverflow.com/a/65425151

* Hide style option on Windows

also exclude native option from dropdown just in case.

* Format

* Minor naming tweak
2023-01-18 21:24:16 +10:00
Damien Elmes
cf45cbf429
Rework syncing code, and replace local sync server (#2329)
This PR replaces the existing Python-driven sync server with a new one in Rust.
The new server supports both collection and media syncing, and is compatible
with both the new protocol mentioned below, and older clients. A setting has
been added to the preferences screen to point Anki to a local server, and a
similar setting is likely to come to AnkiMobile soon.

Documentation is available here: <https://docs.ankiweb.net/sync-server.html>

In addition to the new server and refactoring, this PR also makes changes to the
sync protocol. The existing sync protocol places payloads and metadata inside a
multipart POST body, which causes a few headaches:

- Legacy clients build the request in a non-deterministic order, meaning the
entire request needs to be scanned to extract the metadata.
- Reqwest's multipart API directly writes the multipart body, without exposing
the resulting stream to us, making it harder to track the progress of the
transfer. We've been relying on a patched version of reqwest for timeouts,
which is a pain to keep up to date.

To address these issues, the metadata is now sent in a HTTP header, with the
data payload sent directly in the body. Instead of the slower gzip, we now
use zstd. The old timeout handling code has been replaced with a new implementation
that wraps the request and response body streams to track progress, allowing us
to drop the git dependencies for reqwest, hyper-timeout and tokio-io-timeout.

The main other change to the protocol is that one-way syncs no longer need to
downgrade the collection to schema 11 prior to sending.
2023-01-18 12:43:46 +10:00
Matthias Metelka
9f8667fb47
Auto-hide toolbar in Reviewer (#2262)
* Give webviews a slide-in animation

if reduced motion isn't set.

* Auto-hide toolbar in review mode

moving the mouse above the main webview expands the toolbar. When the mouse leaves the toolbar, it will collapse after a delay of 2s.

* Save some space on bottom toolbars

* Use props for all hard-coded transition durations

and decrease most commonly used duration (200ms) to 150ms.

* Move auto-hide logic into ToolbarWebView

and handle auto-hide specific events in the respective webview subclasses.

* Fix typing issues

* Fix flickering issue

* Add auto_hide_toolbar opt-in to preferences

* Rename hide_toolbar to collapse_toolbar

to better describe the dock-like behaviour.

* Rename setting to minimize_distractions

* Reduce calls to pm in eventFilter

* Run formatter

* Revert setting title to something more specific

* Increase default animation time to 180ms

* Inset toolbar in review mode

when auto-hide is not enabled.

* Use card background on toolbar and add glass effect

* Use flatten/elevate over inset/outset

* Use flatten/elevate over inset/outset

* Update toolbar.py

* Fix toolbar background delay

* Tweak styles

* Use "collapse" instead of "auto-hide"

* Fix background misalignment in collapse mode

* Do not collapse toolbar when pointer is outside MainWebView

* Reduce hide_timer interval to 1000ms

* Use CSS to hide toolbar instead of setting webview height

* Add guard to prevent backdrop-filter: blur on Qt 5.14

* Apply transition to body instead of toolbar

to not complicate things for #2301.

* Fix Qt 5.14 and apply guard globally

* Fix background image scaling difference

* Tweak preference wording (dae)
2023-01-09 14:39:31 +10:00
Abdo
46c0c281f8
Use a webview to show add-on's config help (#2281)
* Use a webview to show add-on's config help

This allows add-ons to embed images for example.

* Improve initial size of splitter widgets

* Decrease font size and margin of webview
2022-12-21 16:55:31 +10:00
Damien Elmes
01f6d618bc Increase initial add-on dialog size
I can't reproduce the reported issue, but this will probably help.

https://forums.ankiweb.net/t/anki-2-1-55-beta-7/25130/10
2022-12-09 12:36:28 +10:00
Damien Elmes
5e0a761b87
Move away from Bazel (#2202)
(for upgrading users, please see the notes at the bottom)

Bazel brought a lot of nice things to the table, such as rebuilds based on
content changes instead of modification times, caching of build products,
detection of incorrect build rules via a sandbox, and so on. Rewriting the build
in Bazel was also an opportunity to improve on the Makefile-based build we had
prior, which was pretty poor: most dependencies were external or not pinned, and
the build graph was poorly defined and mostly serialized. It was not uncommon
for fresh checkouts to fail due to floating dependencies, or for things to break
when trying to switch to an older commit.

For day-to-day development, I think Bazel served us reasonably well - we could
generally switch between branches while being confident that builds would be
correct and reasonably fast, and not require full rebuilds (except on Windows,
where the lack of a sandbox and the TS rules would cause build breakages when TS
files were renamed/removed).

Bazel achieves that reliability by defining rules for each programming language
that define how source files should be turned into outputs. For the rules to
work with Bazel's sandboxing approach, they often have to reimplement or
partially bypass the standard tools that each programming language provides. The
Rust rules call Rust's compiler directly for example, instead of using Cargo,
and the Python rules extract each PyPi package into a separate folder that gets
added to sys.path.

These separate language rules allow proper declaration of inputs and outputs,
and offer some advantages such as caching of build products and fine-grained
dependency installation. But they also bring some downsides:

- The rules don't always support use-cases/platforms that the standard language
tools do, meaning they need to be patched to be used. I've had to contribute a
number of patches to the Rust, Python and JS rules to unblock various issues.
- The dependencies we use with each language sometimes make assumptions that do
not hold in Bazel, meaning they either need to be pinned or patched, or the
language rules need to be adjusted to accommodate them.

I was hopeful that after the initial setup work, things would be relatively
smooth-sailing. Unfortunately, that has not proved to be the case. Things
frequently broke when dependencies or the language rules were updated, and I
began to get frustrated at the amount of Anki development time I was instead
spending on build system upkeep. It's now about 2 years since switching to
Bazel, and I think it's time to cut losses, and switch to something else that's
a better fit.

The new build system is based on a small build tool called Ninja, and some
custom Rust code in build/. This means that to build Anki, Bazel is no longer
required, but Ninja and Rust need to be installed on your system. Python and
Node toolchains are automatically downloaded like in Bazel.

This new build system should result in faster builds in some cases:

- Because we're using cargo to build now, Rust builds are able to take advantage
of pipelining and incremental debug builds, which we didn't have with Bazel.
It's also easier to override the default linker on Linux/macOS, which can
further improve speeds.
- External Rust crates are now built with opt=1, which improves performance
of debug builds.
- Esbuild is now used to transpile TypeScript, instead of invoking the TypeScript
compiler. This results in faster builds, by deferring typechecking to test/check
time, and by allowing more work to happen in parallel.

As an example of the differences, when testing with the mold linker on Linux,
adding a new message to tags.proto (which triggers a recompile of the bulk of
the Rust and TypeScript code) results in a compile that goes from about 22s on
Bazel to about 7s in the new system. With the standard linker, it's about 9s.

Some other changes of note:

- Our Rust workspace now uses cargo-hakari to ensure all packages agree on
available features, preventing unnecessary rebuilds.
- pylib/anki is now a PEP420 implicit namespace, avoiding the need to merge
source files and generated files into a single folder for running. By telling
VSCode about the extra search path, code completion now works with generated
files without needing to symlink them into the source folder.
- qt/aqt can't use PEP420 as it's difficult to get rid of aqt/__init__.py.
Instead, the generated files are now placed in a separate _aqt package that's
added to the path.
- ts/lib is now exposed as @tslib, so the source code and generated code can be
provided under the same namespace without a merging step.
- MyPy and PyLint are now invoked once for the entire codebase.
- dprint will be used to format TypeScript/json files in the future instead of
the slower prettier (currently turned off to avoid causing conflicts). It can
automatically defer to prettier when formatting Svelte files.
- svelte-check is now used for typechecking our Svelte code, which revealed a
few typing issues that went undetected with the old system.
- The Jest unit tests now work on Windows as well.

If you're upgrading from Bazel, updated usage instructions are in docs/development.md and docs/build.md. A summary of the changes:

- please remove node_modules and .bazel
- install rustup (https://rustup.rs/)
- install rsync if not already installed  (on windows, use pacman - see docs/windows.md)
- install Ninja (unzip from https://github.com/ninja-build/ninja/releases/tag/v1.11.1 and
  place on your path, or from your distro/homebrew if it's 1.10+)
- update .vscode/settings.json from .vscode.dist
2022-11-27 15:24:20 +10:00
Matthias Metelka
23e6b2123e
Redesign deck options inputs (#2082)
* Create _input-mixins.scss

* Use button-mixins on more elements

* Replace <select> tag with custom Select component

* Fix RevertButton causing cursor: pointer when hidden

* Increase SaveButton chevron width

* Hide floating component box-shadow when inactive

* Rework SpinBox and move it into components

* Run eslint and prettier

* Remove leftover options prop

* Pass disabled array to EnumSelector again

* Update MapperRow.svelte

* Darken QHeaderView border color

Slipping this in without an extra PR.

* Adjust disabled color, border and cursor

* Remove redundant icon definition from stylesheets

* Fix deck options initial config

* Fix z-index issues in change notetype screen

It might be best to handle z-index locally in each user component instead of hard-coded component values.

* Give web SpinBox a horizontal design

* Give QRadioButton the same treatment as QCheckBox in #2079

* Fix unused CSS selector warning with base button-mixin

* Remove redundant import

* Fix deck options save button

* Delete input-mixins and remove unused down-arrow

* Run eslint on change-notetype

* Run eslint on components
2022-09-27 12:16:45 +10:00
Matthias Metelka
e109c62aa9
Improve hover feedback on various widgets (#2079)
* Use cursor: pointer on QCheckBoxes too and exclude disabled widgets

* Left-align all QCheckBoxes to make hover-area and clickable area the same

Altough the clickable area has always been restricted to the label, the widget itself stretched all the way. This became a problem with the new cursor-pointer for checkboxes.

* Remove Switch duplicate from deck-options

* Add cursor: pointer to Switch and RevertButton

* Add cursor: pointer to bottom toolbar buttons

* Add cursor: pointer to gears

* Add cursor: pointer to radio and checkbox inputs of graphs page

* Improve button appearance in stats screen

* Add cursor: pointer to QTabBar and QToolButton

* Add cursor: pointer to non-editable QComboBox

* Center settings-will-take-effect-after notice in preferences screen

* Use public without_qt5_compat_wrapper() function

* Run prettier
2022-09-20 16:34:15 +10:00
Matthias Metelka
52f52724fa
Add orientation toggle to browser view menu (#2074)
* Use horizontal orientation on browser splitter by default

* Add View menu action to toggle browser orientation

* Add shortcut for toggleOrientation action

based on the most popular add-on.

* Try to fix typing issue

* Make orientation respond to aspect ratio

aspect ratio < 1 means vertical orientation, >= 1 horizontal

* Implement three-way switch for browser orientation

* Fix typing

* Add separator before QWidgetAction

* Use submenu instead of widget and adjust enum

* Add accelerators; move non-accelerator strings into separate .ftl (dae)

* Move BrowserLayout to its own file (dae)
2022-09-20 12:56:59 +10:00
Matthias Metelka
24abb06cee
Color Palette Patch 1 (#2073)
* Make several colors more faint

* Move selection-color definition out of table.py and adjust colors

* Use functions for repetitive border-gradient definitions

* Tweak QTableView and QScrollBar styles

* Use mdi icon for Qt drag handles and adjust padding in browser.ui

* Apply appropriate drag-handle icon depending on orientation

* Fix formatting

* Remove unused import

* Add right margin to browser layout

* Apply splitter styling on macOS as well (dae)

+ Remove the styling from sliders, as it was presumably not intentional.
2022-09-19 10:58:05 +10:00
Matthias Metelka
e2193950a9
Add animation toggle to preferences (#2041)
* Add animation toggle to preferences

and move settings affecting appearance together.

* Add class to body if animations unchecked

* Fix formatting in preferences.ftl

* Update duration(height) function for Collapsible transition

and add explanation.

* Fix formatting

* Increase duration baseline to 10 and decrease factor to 20

* Restore initial layout and rename option to "Reduce motion"

* Move checkboxes together and fix tab order (dae)

+ Remove separation of UI size
2022-09-03 12:14:47 +10:00
Matthias Metelka
d110c4916c
Introduce setting to collapse field by default (#1990)
* Introduce setting to collapse field by default

* Fix schema order

* Change wording from adjective to imperative

sounds a bit less clunky

* Update rslib/src/notetype/schema11.rs (dae)

* Keep settings in single column

* Add back Toggle Visual Editor string

* Add RichTextBadge component and show it conditionally

* Reverse input order depending on default setting

* Make PlainTextInput border-radius responsive to toggle states

* Prevent first Collapsible transition differently

* Focus inputs after Collapsible transition

The double tick calls are just a temporary solution until I find the exact moment an input is focusable again.

* Use requestAnimationFrame to await focusable state

Note: Svelte tick doesn't seem to work in this scenario.
2022-08-31 23:34:39 +10:00
Matthias Metelka
d1cbb86178
Default input setting in fields dialog (#1987)
* Introduce field setting to use plain text editor by default

* Remove leftover function from #1476

* Use boolean instead of string

* Simplify clear_other_field_duplicates

* Convert plain text key to camelCase

* Move HTML item below the existing checkbox, instead of to the right (dae)

Showing it on the right is more space efficient, but feels a bit
cluttered IMHO.
2022-08-18 12:30:18 +10:00
Damien Elmes
d482e90c6b Turn new import/export code on by default
Closes #1966
2022-07-18 13:31:24 +10:00
RumovZ
6da5e5b042
CSV import/export fixes and features (#1898)
* Fix footer moving upwards

* Fix column detection

Was broken because escaped line breaks were not considered.
Also removes delimiter detection on `#columns:` line. User must use tabs
or set delimiter beforehand.

* Add CSV preview

* Parse `#tags column:`

* Optionally export deck and notetype with CSV

* Avoid clones in CSV export

* Prevent bottom of page appearing under footer (dae)

* Increase padding to 1em (dae)

With 0.5em, when a vertical scrollbar is shown, it sits right next to
the right edge of the content, making it look like there's no right
margin.

* Experimental changes to make table fit+scroll (dae)

- limit individual cells to 15em, and show ellipses when truncated
- limit total table width to body width, so that inner table is shown
with scrollbar
- use class rather than id - ids are bad practice in Svelte components,
as more than one may be displayed on a single page

* Skip importing foreign notes with filtered decks

Were implicitly imported into the default deck before.
Also some refactoring to fetch deck ids and names beforehand.

* Hide spacer below hidden field mapping

* Fix guid being replaced when updating note

* Fix dupe identity check

Canonify tags before checking if dupe is identical, but only add update
tags later if appropriate.

* Fix deck export for notes with missing card 1

* Fix note lines starting with `#`

csv crate doesn't support escaping a leading comment char. :(

* Support import/export of guids

* Strip HTML from preview rows

* Fix initially set deck if current is filtered

* Make isHtml toggle reactive

* Fix `html_to_text_line()` stripping sound names

* Tweak export option labels

* Switch to patched rust-csv fork

Fixes writing lines starting with `#`, so revert 5ece10ad05f331.

* List column options with first column field

* Fix flag for exports with HTML stripped
2022-06-09 10:28:01 +10:00
Damien Elmes
9e5fef00d8 Expose new import/export code via prefs checkbox 2022-06-02 16:50:32 +10:00
RumovZ
5f9451f547
Add apkg import/export on backend (#1743)
* Add apkg export on backend

* Filter out missing media-paths at write time

* Make TagMatcher::new() infallible

* Gather export data instead of copying directly

* Revert changes to rslib/src/tags/

* Reuse filename_is_safe/check_filename_safe()

* Accept func to produce MediaIter in export_apkg()

* Only store file folder once in MediaIter

* Use temporary tables for gathering

export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.

* Use schedule_as_new() to reset cards

* ExportData → ExchangeData

* Ignore ascii case when filtering system tags

* search_notes_cards_into_table →

search_cards_of_notes_into_table

* Start on apkg importing on backend

* Fix due dates in days for apkg export

* Refactor import-export/package

- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.

* Add SafeMediaEntry for deserialized MediaEntries

* Prepare media based on checksums

- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.

* Handle encoding in `replace_media_refs()`

* Add trait to keep down cow boilerplate

* Add notetypes immediately instaed of preparing

* Move target_col into Context

* Add notes immediately instaed of preparing

* Note id, not guid of conflicting notes

* Add import_decks()

* decks_configs → deck_configs

* Add import_deck_configs()

* Add import_cards(), import_revlog()

* Use dyn instead of generic for media_fn

Otherwise, would have to pass None with type annotation in the default
case.

* Fix signature of import_apkg()

* Fix search_cards_of_notes_into_table()

* Test new functions in text.rs

* Add roundtrip test for apkg (stub)

* Keep source id of imported cards (or skip)

* Keep source ids of imported revlog (or skip)

* Try to keep source ids of imported notes

* Make adding notetype with id undoable

* Wrap apkg import in transaction

* Keep source ids of imported deck configs (or skip)

* Handle card due dates and original due/did

* Fix importing cards/revlog

Card ids are manually uniquified.

* Factor out card importing

* Refactor card and revlog importing

* Factor out card importing

Also handle missing parents .

* Factor out note importing

* Factor out media importing

* Maybe upgrade scheduler of apkg

* Fix parent deck gathering

* Unconditionally import static media

* Fix deck importing edge cases

Test those edge cases, and add some global test helpers.

* Test note importing

* Let import_apkg() take a progress func

* Expand roundtrip apkg test

* Use fat pointer to avoid propogating generics

* Fix progress_fn type

* Expose apkg export/import on backend

* Return note log when importing apkg

* Fix archived collection name on apkg import

* Add CollectionOpWithBackendProgress

* Fix wrong Interrupted Exception being checked

* Add ClosedCollectionOp

* Add note ids to log and strip HTML

* Update progress when checking incoming media too

* Conditionally enable new importing in GUI

* Fix all_checksums() for media import

Entries of deleted files are nulled, not removed.

* Make apkg exporting on backend abortable

* Return number of notes imported from apkg

* Fix exception printing for QueryOp as well

* Add QueryOpWithBackendProgress

Also support backend exporting progress.

* Expose new apkg and colpkg exporting

* Open transaction in insert_data()

Was slowing down exporting by several orders of magnitude.

* Handle zstd-compressed apkg

* Add legacy arg to ExportAnkiPackage

Currently not exposed on the frontend

* Remove unused import in proto file

* Add symlink for typechecking of import_export_pb2

* Avoid kwargs in pb message creation, so typechecking is not lost

Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign 
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.

* Avoid re-exporting protobuf msgs we only use internally

* Stop after one test failure

mypy often fails much faster than pylint

* Avoid an extra allocation when extracting media checksums

* Update progress after prepare_media() finishes

Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.

* Show progress of note imports

Note import is the slowest part, so showing progress here makes the UI
feel more responsive.

* Reset filtered decks at import time

Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.

We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).

https://github.com/ankitects/anki/pull/1743/files#r839346423

* Fix a corner-case where due dates were shifted by a day

This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.

* Log conflicting note in remapped nt case

* take_fields() → into_fields()

* Alias `[u8; 20]` with `Sha1Hash`

* Truncate logged fields

* Rework apkg note import tests

- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)

* Fix sorting of imported decks

Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.

* target[_id]s → existing_card[_id]s

* export_collection_extracting_media() → ...

export_into_collection_file()

* target_already_exists→card_ordinal_already_exists

* Add search_cards_of_notes_into_table.sql

* Imrove type of apkg export selector/limit

* Remove redundant call to mod_schema()

* Parent tooltips to mw

* Fix a crash when truncating note text

String::truncate() is a bit of a footgun, and I've hit this before
too :-)

* Remove ExportLimit in favour of separate classes

* Remove OpWithBackendProgress and ClosedCollectionOp

Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.

Also fix aborting of colpkg exports, which slipped through in #1817.

* Tidy up import log

* Avoid QDialog.exec()

* Default to excluding scheuling for deck list deck

* Use IncrementalProgress in whole import_export code

* Compare checksums when importing colpkgs

* Avoid registering changes if hashes are not needed

* ImportProgress::Collection → ImportProgress::File

* Make downgrading apkgs depend on meta version

* Generalise IncrementableProgress

And use it in entire import_export code instead.

* Fix type complexity lint

* Take count_map for IncrementableProgress::get_inner

* Replace import/export env with Shift click

* Accept all args from update() for backend progress

* Pass fields of ProgressUpdate explicitly

* Move update_interval into IncrementableProgress

* Outsource incrementing into Incrementor

* Mutate ProgressUpdate in progress_update callback

* Switch import/export legacy toggle to profile setting

Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.

One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.

* Show extension in export dialog

* Continue to provide separate options for schema 11+18 colpkg export

* Default to colpkg export when using File>Export

* Improve appearance of combo boxes when switching between apkg/colpkg

+ Deal with long deck names

* Convert newlines to spaces when showing fields from import

Ensures each imported note appears on a separate line

* Don't separate total note count from the other summary lines

This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.

* Fix 'deck not normal' error when importing a filtered deck for the 2nd time

* Fix [Identical] being shown on first import

* Revert "Continue to provide separate options for schema 11+18 colpkg export"

This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.

Will use a different approach

* Move legacy support into a separate exporter option; add to apkg export

* Adjust 'too new' message to also apply to .apkg import case

* Show a better message when attempting to import new apkg into old code

Previously the user could end seeing a message like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

Unfortunately we can't retroactively fix this for older clients.

* Hide legacy support option in older exporting screen

* Reflect change from paths to fnames in type & name

* Make imported decks normal at once

Then skip special casing in update_deck(). Also skip updating
description if new one is empty.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-05-02 21:12:46 +10:00
roxgib
77d14ddf1e
Add minimumContentsLength property to templatesBox (#1781)
Addresses #1777 by adding the `minimumContentsLength` property to the comboBox, which causes the size policy to not require expanding beyond that amount while still expanding if the user stretches the window manually.

Note that users who were previously experiencing large window widths due to the previous behaviour might see the same size again due the geometry being restored.
2022-04-09 13:55:03 +10:00
Damien Elmes
4515c41d2c
Backup improvements (#1728)
* Collection needs to be closed prior to backup even when not downgrading

* Backups -> BackupLimits

* Some improvements to backup_task

- backup_inner now returns the error instead of logging it, so that
the frontend can discover the issue when they await a backup (or create
another one)
- start_backup() was acquiring backup_task twice, and if another thread
started a backup between the two locks, the task could have been accidentally
overwritten without awaiting it

* Backups no longer require a collection close

- Instead of closing the collection, we ensure there is no active
transaction, and flush the WAL to disk. This means the undo history
is no longer lost on backup, which will be particularly useful if we
add a periodic backup in the future.
- Because a close is no longer required, backups are now achieved with
a separate command, instead of being included in CloseCollection().
- Full sync no longer requires an extra close+reopen step, and we now
wait for the backup to complete before proceeding.
- Create a backup before 'check db'

* Add File>Create Backup

https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157

* Defer checkpoint until we know we need it

When running periodic backups on a timer, we don't want to be fsync()ing
unnecessarily.

* Skip backup if modification time has not changed

We don't want the user leaving Anki open overnight, and coming back
to lots of identical backups.

* Periodic backups

Creates an automatic backup every 30 minutes if the collection has been
modified.

If there's a legacy checkpoint active, tries again 5 minutes later.

* Switch to a user-configurable backup duration

CreateBackup() now uses a simple force argument to determine whether
the user's limits should be respected or not, and only potentially
destructive ops (full download, check DB) override the user's configured
limit.

I considered having a separate limit for collection close and automatic
backups (eg keeping the previous 5 minute limit for collection close),
but that had two downsides:

- When the user closes their collection at the end of the day, they'd
get a recent backup. When they open the collection the next day, it
would get backed up again within 5 minutes, even though not much had
changed.
- Multiple limits are harder to communicate to users in the UI

Some remaining decisions I wasn't 100% sure about:

- If force is true but the collection has not been modified, the backup
will be skipped. If the user manually deleted their backups without
closing Anki, they wouldn't get a new one if the mtime hadn't changed.
- Force takes preference over the configured backup interval - should
we be ignored the user here, or take no backups at all?

Did a sneaky edit of the existing ftl string, as it hasn't been live
long.

* Move maybe_backup() into Collection

* Use a single method for manual and periodic backups

When manually creating a backup via the File menu, we no longer make
the user wait until the backup completes. As we continue waiting for
the backup in the background, if any errors occur, the user will get
notified about it fairly quickly.

* Show message to user if backup was skipped due to no changes

+ Don't incorrectly assert a backup will be created on force

* Add "automatic" to description

* Ensure we backup prior to importing colpkg if collection open

The backup doesn't happen when invoked from 'open backup' in the profile
screen, which matches Anki's previous behaviour. The user could
potentially clobber up to 30 minutes of their work if they exited to
the profile screen and restored a backup, but the alternative is we
create backups every time a backup is restored, which may happen a number
of times if the user is trying various ones. Or we could go back to a
separate throttle amount for this case, at the cost of more complexity.

* Remove the 0 special case on backup interval; minimum of 5 minutes

https://github.com/ankitects/anki/pull/1728#discussion_r830876833
2022-03-21 19:40:42 +10:00
RumovZ
b9c3b12f71
Optionally restore original position and reset counts when forgetting (#1714)
* Add forget prompt with options

- Restore original position
- Reset reps and lapses

* Restore position when resetting for export

* Add config context to avoid passing keys

* Add routine to fetch defaults; use method-specific enum (dae)

* Keep original position by default (dae)

* Fix code completion for forget dialog (dae)

Needs to be a symbolic link to the generated file
2022-03-09 16:51:41 +10:00
RumovZ
f3c8857421
Backups (#1685)
* Add zstd dep

* Implement backend backup with zstd

* Implement backup thinning

* Write backup meta

* Use new file ending anki21b

* Asynchronously backup on collection close in Rust

* Revert "Add zstd dep"

This reverts commit 3fcb2141d2be15f907269d13275c41971431385c.

* Add zstd again

* Take backup col path from col struct

* Fix formatting

* Implement backup restoring on backend

* Normalize restored media file names

* Refactor `extract_legacy_data()`

A bit cumbersome due to borrowing rules.

* Refactor

* Make thinning calendar-based and gradual

* Consider last kept backups of previous stages

* Import full apkgs and colpkgs with backend

* Expose new backup settings

* Test `BackupThinner` and make it deterministic

* Mark backup_path when closing optional

* Delete leaky timer

* Add progress updates for restoring media

* Write restored collection to tempfile first

* Do collection compression in the background thread

This has us currently storing an uncompressed and compressed copy of
the collection in memory (not ideal), but means the collection can be
closed without waiting for compression to complete. On a large collection,
this takes a close and reopen from about 0.55s to about 0.07s. The old
backup code for comparison: about 0.35s for compression off, about
8.5s for zip compression.

* Use multithreading in zstd compression

On my system, this reduces the compression time of a large collection
from about 0.55s to 0.08s.

* Stream compressed collection data into zip file

* Tweak backup explanation

+ Fix incorrect tab order for ignore accents option

* Decouple restoring backup and full import

In the first case, no profile is opened, unless the new collection
succeeds to load.
In the second case, either the old collection is reloaded or the new one
is loaded.

* Fix number gap in Progress message

* Don't revert backup when media fails but report it

* Tweak error flow

* Remove native BackupLimits enum

* Fix type annotation

* Add thinning test for whole year

* Satisfy linter

* Await async backup to finish

* Move restart disclaimer out of backup tab

Should be visible regardless of the current tab.

* Write restored collection in chunks

* Refactor

* Write media in chunks and refactor

* Log error if removing file fails

* join_backup_task -> await_backup_completion

* Refactor backup.rs

* Refactor backup meta and collection extraction

* Fix wrong error being returned

* Call sync_all() on new collection

* Add ImportError

* Store logger in Backend, instead of creating one on demand

init_backend() accepts a Logger rather than a log file, to allow other
callers to customize the logger if they wish.

In the future we may want to explore using the tracing crate as an
alternative; it's a bit more ergonomic, as a logger doesn't need to be
passed around, and it plays more nicely with async code.

* Sync file contents prior to rename; sync folder after rename.

* Limit backup creation to once per 30 min

* Use zstd::stream::copy_decode

* Make importing abortable

* Don't revert if backup media is aborted

* Set throttle implicitly

* Change force flag to minimum_backup_interval

* Don't attempt to open folders on Windows

* Join last backup thread before starting new one

Also refactor.

* Disable auto sync and backup when restoring again

* Force backup on full download

* Include the reason why a media file import failed, and the file path

- Introduce a FileIoError that contains a string representation of
the underlying I/O error, and an associated path. There are a few
places in the code where we're currently manually including the filename
in a custom error message, and this is a step towards a more consistent
approach (but we may be better served with a more general approach in
the future similar to Anyhow's .context())
- Move the error message into importing.ftl, as it's a bit neater
when error messages live in the same file as the rest of the messages
associated with some functionality.

* Fix importing of media files

* Minor wording tweaks

* Save an allocation

I18n strings with replacements are already strings, so we can skip the
extra allocation. Not that it matters here at all.

* Terminate import if file missing from archive

If a third-party tool is creating invalid archives, the user should know
about it. This should be rare, so I did not attempt to make it
translatable.

* Skip multithreaded compression on small collections

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-03-07 15:11:31 +10:00
Henrik Giesel
76bdfb3899
Add a flag to allow loading the fields dialog in a web view (#1706)
* Load fields_web from fields.py if appropriate flag is set

* Add FieldsPage as entry for new fields view

* Pass mypy

* Fix pylint

* Fix fields_web in Qt5 (dae)

May not be related to the CI error, but required for compatibility
with Qt5.
2022-03-03 22:45:26 +10:00
RumovZ
700fc50f7a
View menu (#1668)
* Add main view menu

* Add browser view menu

* Use standard keys for zooming and full screen

* Capitalise menu item names

* Toggle Showing Cards/Notes -> Toggle Cards/Notes

* Explicitly set linux full screen key

on_toggle_fullscreen -> on_toggle_full_screen
2022-02-17 16:31:46 +10:00
Abdo
fd6a78e7cf
Add option to ignore accents in search by default (#1667)
* Add option to ignore accents in unqualified search

* Support ignoring accents in word boundary search

* Update ftl/core/preferences.ftl
2022-02-17 16:30:52 +10:00
RumovZ
cc91017bf1
Remove unused actions from ui files (#1655)
* Remove redundant actions in add dialog

* Remove unused actions from main and browser
2022-02-11 10:15:36 +10:00
Damien Elmes
b64c179609 change notes/cards toggle shortcut to ctrl+alt+t
Closes #1652
2022-02-09 12:59:45 +10:00
Damien Elmes
369223ce8b fix generated code symlinks again
To work with f842ab7c9d
2022-01-24 16:59:37 +10:00
Damien Elmes
5efaf5a4be move Bazel convenience symlinks outside of repo folder
The default symlink location can cause slowdowns and wasted CPU cycles
in VS Code and PyCharm/IntelliJ, as they try to watch Bazel's (large)
build folder for changes. The issue can be mostly ameliorated in VS Code
by excluding the symlinks using globs in settings like watcherExclude,
but the Rust extension doesn't support globs, so each folder needs to be
listed out separately. And because the product name symlink depends on
the name of the directory you're building from, we can't just include
the excludes in .vscode - it will depend on the folder the user is storing
things.

PyCharm and IntelliJ behave even worse here - they continue to monitor
for changes in all folders of the repo, even if those folders have been
marked as excluded in the project settings. Placing the folders into the
IDE-global Editor>File Types>Ignored Files And Folders works around this,
but again we run into troubles making this work out of the box, especially
with the product name in the symlink.

One option would be to turn the symlinks off completely. They are not
required for building, and for scripting/debugging, we can get the folder
locations via 'bazel info'. But with that approach, we would no longer
be able to symlink build products into the source tree, as we do for
things like the generated backend methods and translations, so we'd lose
code completion for them that way.

Another option would be to place the symlinks in .bazel/ inside the repo.
That solves the VS Code case (in conjunction with a workspace config file),
but doesn't fully fix IntelliJ/PyCharm.

The only remaining option I can see is to place the symlinks outside the
repo. Bazel won't expand ~ in the symlink path, so we can't use something
like ~/.cache/bazel/anki to place the files near the other build files.
So we end up having to have the files written to ../bazel/anki, in the
repo's parent folder. Not very clean, but I don't see a better alternative
at the moment.

.gitignore is still ignoring bazel-*, as currently bazel-dist and
bazel-pkg will be created when building/packaging. They should be fairly
innocuous, but we may want to rename them at one point.

Other changes:

- add missing symlink for pylib hooks
- add a sample .user.bazelrc file
2022-01-23 19:18:44 +10:00
RumovZ
65f6728454
Fix Alt-Gr and add shortcut to create copy again (#1544)
* Set QT_QPA_PLATFORM for Win on recent Qt versions

Erroneously dropped for Qt 5.15.1+.

* Enable original shortcut to create copy

* minor tweaks to comment to make it clearer (dae)
2021-12-10 17:52:08 +10:00
Damien Elmes
7ef40cb850 disable 'create copy' shortcut for now
In preparation for a new beta; we can add a new shortcut once one is
decided on https://github.com/ankitects/anki/pull/1541
2021-12-09 10:24:21 +10:00
RumovZ
cd22485f9b
Add browser action to create note copy (#1535)
* Add browser action to create note copy

* Use new note and copy instead of using source

* Change shortcut due to Qt's Alt-Gr bug

* Add separate routine for suffixing action with ...

* Remove '...' from some translations

The convention is to add an ellipsis if more input is required to
perform the action. Whether or not the action opens a new window is not
decisive.
Sources:
https://developer.apple.com/design/human-interface-guidelines/macos/menus/menu-anatomy/
https://docs.microsoft.com/en-us/windows/win32/uxguide/cmd-toolbars
2021-12-08 08:40:48 +10:00
RumovZ
f2173fddb0
Live theme changes (#1497)
* Allow theme change at runtime and add hook

* Save or restore default palette on theme change

* Update aqt widget styles on theme change

* styling fixes

- drop _light_palette, as default_palette serves the same purpose
- save default platform theme, and restore it when switching away
from nightmode
- update macOS light/dark mode on theme switch
- fix unreadable menus on Windows

* update night-mode classes on theme change

This is the easy part - CSS styling that uses standard_css or our
css variables should update automatically. The main remaining issue
is JS code that sets colors based on the theme at the time it's run -
eg the graph code, and the editor.

* switch night mode value on toggle

* expose current theme via a store; switch graphs to use it

https://github.com/ankitects/anki/issues/1471#issuecomment-972402492

* start using currentTheme in editor/components

This fixes basic editing - there are still components that need updating.

* add simple xcodeproj for code completion

* add helper to get currently-active system theme on macOS

* fix setCurrentTheme not being immediately available

* live update tag color

* style().name() doesn't work on Qt5

* automatic theme switching on Windows/Mac

* currentTheme -> pageTheme

* Replace `nightModeKey` with `pageTheme`

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2021-11-25 07:17:41 +10:00
Damien Elmes
18b7691e2c drop beta tag from v3 scheduler
Will exit beta when 2.1.50 is released as stable
2021-11-23 11:10:04 +10:00
RumovZ
63316f763d
Make add cards dialog a main window (#1486)
Closes #980.
2021-11-14 11:33:59 +10:00
Matthias Metelka
371f731e30
Editor Field Descriptions (#1476)
* Add description input to fields dialog

QLineEdit seems like the best option, as it saves space and motivates users to keep their descriptions concise.

* Add setDescriptions to note initialization script

Went for the extra function instead of including it in setFields to prevent potential add-on breakages.

* Add tooltip next to field name if description is set

* Refactor code according to suggestions

Set default tooltip placement to right instead of bottom

Use .get() for fld["description"]

Fix tab order in fields dialog

Swap out abbreviation "desc" for full length name to keep consistency

* Update Protobuf and Rust for description

Add description to notetypes.proto and schema11

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>

* Fix tooltips not updating with description

Remove redundant variable tooltipOptions

Update previousTooltip within reactive function

* Move LabelDescription out of LabelName

Co-authored-by: Henrik Giesel <hgiesel@users.noreply.github.com>

* Decrease icon size and fix alignment

Co-Authored-By: Henrik Giesel <hengiesel@gmail.com>

* the new key needs to be cleared from fields, not the notetype itself

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>
Co-authored-by: Henrik Giesel <hengiesel@gmail.com>
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2021-11-06 09:42:48 +10:00
Damien Elmes
8cd905a35f remove a few unnecessary PyQt workarounds
https://www.riverbankcomputing.com/pipermail/pyqt/2021-October/044314.html
2021-10-24 14:24:35 +10:00
Damien Elmes
ee644e08a3 fixes and documentation for Linux ARM64
+ add qt6 dep to wheel install docs
+ remove x86_64 constraint on orjson
2021-10-23 15:22:24 +10:00
Damien Elmes
a14eb6a1e8 improve PyQt install
- use a single script for all PyQt versions
- add hashes
- add a new ./run-qt5.14 script for testing with PyQt5.14
2021-10-23 10:56:17 +10:00
Damien Elmes
281f1b2bf9 build all the UI files in one go
On macOS, the overhead of importing PyQt for each file far exceeds
any gains we get from incremental recompilation.
2021-10-16 09:20:20 +10:00
Damien Elmes
2c17105a35 use string replacements to generate Qt5 forms
They are mostly compatible, and this means a Qt5 install is not required
as part of the build process.
2021-10-16 08:59:09 +10:00
Damien Elmes
52642d693b drop PyAudio support
I do not recall anyone reporting that it worked better than the Qt
implementation for them, and the lack of recent wheels on PyPI is a pain.
We can always add it back in the future if enough people come out of
the woodwork to report they were using it.
2021-10-15 13:15:16 +10:00
Damien Elmes
a7812dedc0 switch to new-style PyQt scoped enums and Qt6
The enum changes should work on PyQt 5.x, and are required in PyQt 6.x.
They are not supported by the PyQt5 typings however, so we need to run
our tests with PyQt6.
2021-10-15 12:57:19 +10:00
Damien Elmes
b0272f0336 generate pyqt6 forms 2021-10-15 12:57:19 +10:00
evandrocoan
2a12d44b34
Increased max visible items from clayout_top.ui (#1425)
10 is too small for big decks.
2021-10-14 19:36:04 +10:00
Damien Elmes
e357dbf6b3 use Qt search path instead of resource system
Means URLs like :/icons/foo.jpg should become icons:foo.jpg

This is part of the prep work for a PyQt6 update. PyQt6 has dropped
pyrcc, so we can longer generate the icons_qrc.py file we did previously.

Qt Designer expects us to use the resource system, so we continue to
generate the icons.qrc file to make editing the UI files easier. But at
runtime, we no longer use that file.
2021-10-12 16:17:08 +10:00
Damien Elmes
546d32cfda ctrl+alt+n for forget
https://forums.ankiweb.net/t/anki-2-1-44-make-cards-new/12247/7
2021-08-18 15:32:45 +10:00
Damien Elmes
40c1a86a25 add symlink to generated icons.qrc
Otherwise Qt Designer tries to remove the reference.
2021-08-18 15:32:45 +10:00
RumovZ
a5193339e7 Rework Find & Replace dialog:
- Add option to affect whole collection
- Allow to open without selection
- Add parameter for presetting field
2021-07-13 16:33:45 +02:00
Henrik Giesel
8edcc54aae Add flag icons from mdi after all
- This way, we can also use their modifier versions with little pluses/minuses
2021-07-01 01:16:50 +02:00