(for upgrading users, please see the notes at the bottom)
Bazel brought a lot of nice things to the table, such as rebuilds based on
content changes instead of modification times, caching of build products,
detection of incorrect build rules via a sandbox, and so on. Rewriting the build
in Bazel was also an opportunity to improve on the Makefile-based build we had
prior, which was pretty poor: most dependencies were external or not pinned, and
the build graph was poorly defined and mostly serialized. It was not uncommon
for fresh checkouts to fail due to floating dependencies, or for things to break
when trying to switch to an older commit.
For day-to-day development, I think Bazel served us reasonably well - we could
generally switch between branches while being confident that builds would be
correct and reasonably fast, and not require full rebuilds (except on Windows,
where the lack of a sandbox and the TS rules would cause build breakages when TS
files were renamed/removed).
Bazel achieves that reliability by defining rules for each programming language
that define how source files should be turned into outputs. For the rules to
work with Bazel's sandboxing approach, they often have to reimplement or
partially bypass the standard tools that each programming language provides. The
Rust rules call Rust's compiler directly for example, instead of using Cargo,
and the Python rules extract each PyPi package into a separate folder that gets
added to sys.path.
These separate language rules allow proper declaration of inputs and outputs,
and offer some advantages such as caching of build products and fine-grained
dependency installation. But they also bring some downsides:
- The rules don't always support use-cases/platforms that the standard language
tools do, meaning they need to be patched to be used. I've had to contribute a
number of patches to the Rust, Python and JS rules to unblock various issues.
- The dependencies we use with each language sometimes make assumptions that do
not hold in Bazel, meaning they either need to be pinned or patched, or the
language rules need to be adjusted to accommodate them.
I was hopeful that after the initial setup work, things would be relatively
smooth-sailing. Unfortunately, that has not proved to be the case. Things
frequently broke when dependencies or the language rules were updated, and I
began to get frustrated at the amount of Anki development time I was instead
spending on build system upkeep. It's now about 2 years since switching to
Bazel, and I think it's time to cut losses, and switch to something else that's
a better fit.
The new build system is based on a small build tool called Ninja, and some
custom Rust code in build/. This means that to build Anki, Bazel is no longer
required, but Ninja and Rust need to be installed on your system. Python and
Node toolchains are automatically downloaded like in Bazel.
This new build system should result in faster builds in some cases:
- Because we're using cargo to build now, Rust builds are able to take advantage
of pipelining and incremental debug builds, which we didn't have with Bazel.
It's also easier to override the default linker on Linux/macOS, which can
further improve speeds.
- External Rust crates are now built with opt=1, which improves performance
of debug builds.
- Esbuild is now used to transpile TypeScript, instead of invoking the TypeScript
compiler. This results in faster builds, by deferring typechecking to test/check
time, and by allowing more work to happen in parallel.
As an example of the differences, when testing with the mold linker on Linux,
adding a new message to tags.proto (which triggers a recompile of the bulk of
the Rust and TypeScript code) results in a compile that goes from about 22s on
Bazel to about 7s in the new system. With the standard linker, it's about 9s.
Some other changes of note:
- Our Rust workspace now uses cargo-hakari to ensure all packages agree on
available features, preventing unnecessary rebuilds.
- pylib/anki is now a PEP420 implicit namespace, avoiding the need to merge
source files and generated files into a single folder for running. By telling
VSCode about the extra search path, code completion now works with generated
files without needing to symlink them into the source folder.
- qt/aqt can't use PEP420 as it's difficult to get rid of aqt/__init__.py.
Instead, the generated files are now placed in a separate _aqt package that's
added to the path.
- ts/lib is now exposed as @tslib, so the source code and generated code can be
provided under the same namespace without a merging step.
- MyPy and PyLint are now invoked once for the entire codebase.
- dprint will be used to format TypeScript/json files in the future instead of
the slower prettier (currently turned off to avoid causing conflicts). It can
automatically defer to prettier when formatting Svelte files.
- svelte-check is now used for typechecking our Svelte code, which revealed a
few typing issues that went undetected with the old system.
- The Jest unit tests now work on Windows as well.
If you're upgrading from Bazel, updated usage instructions are in docs/development.md and docs/build.md. A summary of the changes:
- please remove node_modules and .bazel
- install rustup (https://rustup.rs/)
- install rsync if not already installed (on windows, use pacman - see docs/windows.md)
- install Ninja (unzip from https://github.com/ninja-build/ninja/releases/tag/v1.11.1 and
place on your path, or from your distro/homebrew if it's 1.10+)
- update .vscode/settings.json from .vscode.dist
* Prevent global focus border on tag-input
* Fix margin issues with tag editor
* Remove redundant autocomplete call
that caused the addition of an empty tag when tag suggestions were selected with the Enter key.
* Prevent input text from overlapping with newly added tags
... at least when they're selected from the autocomplete list via mouse. If they're selected via keyboard, there's still an overlapping issue.
* Fix error on updateSuggestions
* Hide empty tag
* Make double-click to collapse/expand translatable
* Redesign deck config, swap tooltips for help modals, link to manual
* Replace canvas-inset with canvas-code for custom scheduling
* Make section header link to manual too
* Include elevation Sass library
* Remove two unused exports
* Fix tabbed spinboxes
* Update ftl/core/deck-config.ftl
* Update ftl/core/deck-config.ftl
* Fix format
* Make border-radius and box-shadow more subtle
* Fix margin for vertical aspect ratio
* Make direct hover on info badge apply effect instantly
* Add redirect line to manual underneath chapter
* Add crate snafu
* Replace all inline structs in AnkiError
* Derive Snafu on AnkiError
* Use snafu for card type errors
* Use snafu whatever error for InvalidInput
* Use snafu for NotFoundError and improve message
* Use snafu for FileIoError to attach context
Remove IoError.
Add some context-attaching helpers to replace code returning bare
io::Errors.
* Add more context-attaching io helpers
* Add message, context and backtrace to new snafus
* Utilize error context and backtrace on frontend
* Rename LocalizedError -> BackendError.
* Remove DocumentedError.
* Have all backend exceptions inherit BackendError.
* Rename localized(_description) -> message
* Remove accidentally committed experimental trait
* invalid_input_context -> ok_or_invalid
* ensure_valid_input! -> require!
* Always return `Err` from `invalid_input!`
Instead of a Result to unwrap, the macro accepts a source error now.
* new_tempfile_in_parent -> new_tempfile_in_parent_of
* ok_or_not_found -> or_not_found
* ok_or_invalid -> or_invalid
* Add crate convert_case
* Use unqualified lowercase type name
* Remove uses of snafu::ensure
* Allow public construction of InvalidInputErrors (dae)
Needed to port the AnkiDroid changes.
* Make into_protobuf() public (dae)
Also required for AnkiDroid. Not sure why it worked previously - possible
bug in older Rust version?
* Use horizontal orientation on browser splitter by default
* Add View menu action to toggle browser orientation
* Add shortcut for toggleOrientation action
based on the most popular add-on.
* Try to fix typing issue
* Make orientation respond to aspect ratio
aspect ratio < 1 means vertical orientation, >= 1 horizontal
* Implement three-way switch for browser orientation
* Fix typing
* Add separator before QWidgetAction
* Use submenu instead of widget and adjust enum
* Add accelerators; move non-accelerator strings into separate .ftl (dae)
* Move BrowserLayout to its own file (dae)
* Show warning if multiple type boxes are used
* Report templates referencing media in Media Check
* Apply suggestions from code review
* Fix media-check.ftl
* Only report media references with fields
Like `<img src={{Front}}>`.
Also report Anki sound tags and latex.
* Loop existing media regexes
* Add animation toggle to preferences
and move settings affecting appearance together.
* Add class to body if animations unchecked
* Fix formatting in preferences.ftl
* Update duration(height) function for Collapsible transition
and add explanation.
* Fix formatting
* Increase duration baseline to 10 and decrease factor to 20
* Restore initial layout and rename option to "Reduce motion"
* Move checkboxes together and fix tab order (dae)
+ Remove separation of UI size
* Introduce setting to collapse field by default
* Fix schema order
* Change wording from adjective to imperative
sounds a bit less clunky
* Update rslib/src/notetype/schema11.rs (dae)
* Keep settings in single column
* Add back Toggle Visual Editor string
* Add RichTextBadge component and show it conditionally
* Reverse input order depending on default setting
* Make PlainTextInput border-radius responsive to toggle states
* Prevent first Collapsible transition differently
* Focus inputs after Collapsible transition
The double tick calls are just a temporary solution until I find the exact moment an input is focusable again.
* Use requestAnimationFrame to await focusable state
Note: Svelte tick doesn't seem to work in this scenario.
* Adjust size of legacy buttons
* Revert "Adjust size of legacy buttons"
This reverts commit fb888fe1db9050c34b1a7b0820e6da5ac91ccee6.
* Remove unused function from #1476
* Use outline version for tag icon
* Add chevron icons
* Remove code icons, keep one pin icon version
* Add code-bg color
* Redesign fields
* Remove unused import
* Fix imports
* Move PlainTextBadge between editing inputs
where it belongs :)
* Make whole separator line clickable
* Fix transition
and format
* Don't show toggle when field is collapsed
* Show toggle only on hover
for mobile I'd like to implement a swipe mechanism.
* Use tweened SVG for triangle instead of CSS hack
* Implement more obvious HTML toggle on bottom right
* Reduce field height by a few pixels
* Reduce field height by two pixels
* Show HTML toggle when PlainTextInput is active, regardless of hover/focus
* Remove RichTextBadge.svelte
* Create separate collapsed field state
this means users can collapse fields with the HTML editor open and it will stay open when the field is expanded again.
* Add slide out animation to EditingArea, RichTextInput and PlainTextInput
only for collapsing, because it is choppy on expansion (common issue with Svelte transitions).
* Fix aliasing issue on focused field corners
* Make StickyBadge feel more responsive
* Move StickyBadge closer to field border
* Adjust field gutter/margins
* Make LabelContainer sticky
to make field operations accessible on fields with a lot of content.
* Add back html icons, remove visual editor icons
* Revert "Add code-bg color"
This reverts commit 4200f354193710b3acd9bcf84b67958e200ddcdb.
* Add rich text icon, remove strikethrough code icon
* Revert PlainTextBadge to original position
* Adjust margins in FieldState
* Rename PlainTextBadge to SecondaryInputBadge
in preparation for #1987
* Run eslint and prettier
* Make whole LabelContainer clickable area for collapse/expand
* Revert "Add slide out animation to EditingArea, RichTextInput and PlainTextInput"
This reverts commit 9a2b3410d0ead37ae1da408d68e14507a058a613.
* Fix error on collapse/expansion
this was caused by the {#if} blocks, which resulted in the deletion of original EditingAreas.
* Refocus when toggling chevron and secondary input badge
* Revert "Revert "Add code-bg color""
This reverts commit 1cfd3bda65354ab90c1ab4cbbef47596a1be8754.
* Use single rotating chevron icon and make it RTL-compatible
* Remove redundant CSS transition rule
* Introduce animated Collapsible component and fix refocus on toggle
* Do not try to force repaint, as it is not required
* Remove RTL store from LabelContainer
the direction is already applied globally.
* Collapse secondary input with field
* Add focusedField to NoteEditorAPI
* Replace :global CSS selector with class .visible
thus removing the assumption that the component is used inside an EditorField.
https://github.com/ankitects/anki/pull/2002#discussion_r944876448
* Use named function syntax instead of function expressions
* Add explanation comment
* Remove unnecessary :bind directive
* Create CollapseBadge component
* Move :global selector into .plain-text-input
* Add comment explaining box-shadow pseudo-element
* Move Collapsible from EditingArea, PlainTextInput and RichTextInput into user components
* Rename SecondaryInputBadge to PlainTextBadge and remove generalization logic
I kept the rich text icon inside icons.ts for future use.
* Sort imports
* Fix background-color for duplicates not showing
with yet another pseudo-element :)
The pseudo-element that covers up field borders on scroll caused this issue. Fighting fire with fire here.
* Increase size of plain text toggle to original value again
This makes the clickable area a bit bigger and looks slightly more consistent with StickyBadge.
* Scrap pseudo-element mess in LabelContainer and tackle the actual issue
* Add class .visible to StickyBadge too
This introduces a peculiar bug: The active prop of StickyBadge resets to false when the mouse leaves the field - regardless of the actual back-end value.
* Fix sticky badge resetting on mouseleave/blur
* Apply overflow: hidden only during transition
fixes MathJax handle getting cut off by fields
* Remove unused variable
* Fix visual bug caused by overflow:hidden not applying in time
I tried several asynchronous approaches, but they all caused issues: either they prevented the CSS transition or they made field inputs lose focus.
In the end I resorted to direct, synchronous DOM-manipulation and added an explanatory comment.
* Decrease Collapsible load time by blocking first transition
I noticed the sliding animation has a hefty performance impact when a large number of fields is loaded simultaneously.
Blocking the first transition (which isn't even visible) results in a big boost in load time.
* Replace usages of gap with margins for children
* Revert unnecessary removal of grid-gap definition
* Correct comments about flex-gap property
mistook that for grid-gap.
* Resolve style issues
* Add minimum targets to gap comment
Co-authored-by: Henrik Giesel <hengiesel@gmail.com>
* Introduce field setting to use plain text editor by default
* Remove leftover function from #1476
* Use boolean instead of string
* Simplify clear_other_field_duplicates
* Convert plain text key to camelCase
* Move HTML item below the existing checkbox, instead of to the right (dae)
Showing it on the right is more space efficient, but feels a bit
cluttered IMHO.
* Add deck-specific limits to DeckNormal
* Add deck-specific limits to schema11
* Add DeckLimitsDialog
* deck_limits_qt6.py needs to be a symlink
* Clear duplicate deck setting keys on downgrade
* Export deck limits when exporting with scheduling
* Revert "deck_limits_qt6.py needs to be a symlink"
This reverts commit 4ee7be1e10c4e8c49bb20de3bf45ac18b5e2d4f6.
* Revert "Add DeckLimitsDialog"
This reverts commit eb0e2a62d33df0b518d9204a27b09e97966ce82a.
* Add day limits to DeckNormal
* Add deck and day limits mock to deck options
* Revert "Add deck and day limits mock to deck options"
This reverts commit 0775814989e8cb486483d06727b1af266bb4513a.
* Add Tabs component for daily limits
* Add borders to tabs component
* Revert "Add borders to tabs component"
This reverts commit aaaf5538932540f944d92725c63bb04cfe97ea14.
* Implement tabbed limits properly
* Add comment to translations
* Update rslib/src/decks/limits.rs
Co-authored-by: Damien Elmes <dae@users.noreply.github.com>
* Fix camel case in clear_other_duplicates()
* day_limit → current_limit
* Also import day limits
* Remember last used day limits
* Add day limits to schema 11
* Tweak comment (dae)
* Exclude day limit in export (dae)
* Tweak tab wording (dae)
* Update preset limits on preset change
* Explain tabs in tooltip (dae)
* Omit deck and today limits if v2 is enabled
* Preserve deck limit when switching to today limit
* Throw error for unknown condition fields as well
So if 'foo' is not a field, refuse to save a template containing
`{{#foo}}bar{{/foo}}`. Previously, only `{{foo}}` would be checked.
As a side effect, templates which *only* contain fields as conditions
may be saved. Meh.
* Display template errors in q/a columns only
So the affected browser row remains active and the user can fix the
template more easily.
* Specify if error occured in a browser template
* Minor wording tweak (dae)
There's an argument for using the exact wording as well, but this just
reads a little more naturally to me.
* Move field description into EditingArea as placeholder
* Prevent insertion of breaks into empty fields
to allow :empty CSS selector to also work on fields other than the first one.
* Remove redundant setContext from EditingArea
* Fix import order
* Revert "Prevent insertion of breaks into empty fields"
This reverts commit 1615fd5cf441b1047dae6a34265acb9c5cae50b2.
* Use class:empty instead of :empty CSS pseudo-class
* Restrict description to single line, ellipse overflow
* Make description in field dialog a bit clearer
* Fix footer moving upwards
* Fix column detection
Was broken because escaped line breaks were not considered.
Also removes delimiter detection on `#columns:` line. User must use tabs
or set delimiter beforehand.
* Add CSV preview
* Parse `#tags column:`
* Optionally export deck and notetype with CSV
* Avoid clones in CSV export
* Prevent bottom of page appearing under footer (dae)
* Increase padding to 1em (dae)
With 0.5em, when a vertical scrollbar is shown, it sits right next to
the right edge of the content, making it look like there's no right
margin.
* Experimental changes to make table fit+scroll (dae)
- limit individual cells to 15em, and show ellipses when truncated
- limit total table width to body width, so that inner table is shown
with scrollbar
- use class rather than id - ids are bad practice in Svelte components,
as more than one may be displayed on a single page
* Skip importing foreign notes with filtered decks
Were implicitly imported into the default deck before.
Also some refactoring to fetch deck ids and names beforehand.
* Hide spacer below hidden field mapping
* Fix guid being replaced when updating note
* Fix dupe identity check
Canonify tags before checking if dupe is identical, but only add update
tags later if appropriate.
* Fix deck export for notes with missing card 1
* Fix note lines starting with `#`
csv crate doesn't support escaping a leading comment char. :(
* Support import/export of guids
* Strip HTML from preview rows
* Fix initially set deck if current is filtered
* Make isHtml toggle reactive
* Fix `html_to_text_line()` stripping sound names
* Tweak export option labels
* Switch to patched rust-csv fork
Fixes writing lines starting with `#`, so revert 5ece10ad05f331.
* List column options with first column field
* Fix flag for exports with HTML stripped
* Add crate csv
* Add start of csv importing on backend
* Add Menomosyne serializer
* Add csv and json importing on backend
* Add plaintext importing on frontend
* Add csv metadata extraction on backend
* Add csv importing with GUI
* Fix missing dfa file in build
Added compile_data_attr, then re-ran cargo/update.py.
* Don't use doubly buffered reader in csv
* Escape HTML entities if CSV is not HTML
Also use name 'is_html' consistently.
* Use decimal number as foreign ease (like '2.5')
* ForeignCard.ivl → ForeignCard.interval
* Only allow fixed set of CSV delimiters
* Map timestamp of ForeignCard to native due time
* Don't trim CSV records
* Document use of empty strings for defaults
* Avoid creating CardGenContexts for every note
This requires CardGenContext to be generic, so it works both with an
owned and borrowed notetype.
* Show all accepted file types in import file picker
* Add import_json_file()
* factor → ease_factor
* delimter_from_value → delimiter_from_value
* Map columns to fields, not the other way around
* Fallback to current config for csv metadata
* Add start of new import csv screen
* Temporary fix for compilation issue on Linux/Mac
* Disable jest bazel action for import-csv
Jest fails with an error code if no tests are available, but this would
not be noticable on Windows as Jest is not run there.
* Fix field mapping issue
* Revert "Temporary fix for compilation issue on Linux/Mac"
This reverts commit 21f8a261408cdae49ec031aa21a1b659c4f66d82.
* Add HtmlSwitch and move Switch to components
* Fix spacing and make selectors consistent
* Fix shortcut tooltip
* Place import button at the top with path
* Fix meta column indices
* Remove NotetypeForString
* Fix queue and type of foreign cards
* Support different dupe resolution strategies
* Allow dupe resolution selection when importing CSV
* Test import of unnormalized text
Close #1863.
* Fix logging of foreign notes
* Implement CSV exports
* Use db_scalar() in notes_table_len()
* Rework CSV metadata
- Notetypes and decks are either defined by a global id or by a column.
- If a notetype id is provided, its field map must also be specified.
- If a notetype column is provided, fields are now mapped by index
instead of name at import time. So the first non-meta column is used for
the first field of every note, regardless of notetype. This makes
importing easier and should improve compatiblity with files without a
notetype column.
- Ensure first field can be mapped to a column.
- Meta columns must be defined as `#[meta name]:[column index]` instead
of in the `#columns` tag.
- Column labels contain the raw names defined by the file and must be
prettified by the frontend.
* Adjust frontend to new backend column mapping
* Add force flags for is_html and delimiter
* Detect if CSV is HTML by field content
* Update dupe resolution labels
* Simplify selectors
* Fix coalescence of oneofs in TS
* Disable meta columns from selection
Plus a lot of refactoring.
* Make import button stick to the bottom
* Write delimiter and html flag into csv
* Refetch field map after notetype change
* Fix log labels for csv import
* Log notes whose deck/notetype was missing
* Fix hiding of empty log queues
* Implement adding tags to all notes of a csv
* Fix dupe resolution not being set in log
* Implement adding tags to updated notes of a csv
* Check first note field is not empty
* Temporary fix for build on Linux/Mac
* Fix inverted html check (dae)
* Remove unused ftl string
* Delimiter → Separator
* Remove commented-out line
* Don't accept .json files
* Tweak tag ftl strings
* Remove redundant blur call
* Strip sound and add spaces in csv export
* Export HTML by default
* Fix unset deck in Mnemosyne import
Also accept both numbers and strings for notetypes and decks in JSON.
* Make DupeResolution::Update the default
* Fix missing dot in extension
* Make column indices 1-based
* Remove StickContainer from TagEditor
Fixes line breaking, border and z index on ImportCsvPage.
* Assign different key combos to tag editors
* Log all updated duplicates
Add a log field for the true number of found notes.
* Show identical notes as skipped
* Split tag-editor into separate ts module (dae)
* Add progress for CSV export
* Add progress for text import
* Tidy-ups after tag-editor split (dae)
- import-csv no longer depends on editor
- remove some commented lines
In v3, it's more informative to show the count of child decks separately,
since increasing the limit of the current deck does not increase the limits
of child decks. When we rework the decks list in the future, a tooltip
will hopefully provide an easier way for users to see where cards are
available, and where limits are being applied.
Closes#1868
* Add ResizableImage.svelte in ts/editable
* Set image constrained via attributes instead of managed style sheet
* Implement new constrained size method
* Turn WithImageConstrained.svelte into normal ts file
* Introduce removeStyleProperties
Removes "style" attribute if all style properties were cleared
* Avoid --editor-width and use one variable set on container
* Disable shrinking if already smaller than shrunken size
* Add button to restore image to original size
* Don't allow restoring original size if no custom width set
* Bottom-center HandleLabel
* Satisfy svelte-check
The old `media_files_did_export` hook has been kept around for use with
the legacy apkg exporter (an add-on uses it), and a new
`legacy_export_progress` hook has been added so we can get progress
from the new colpkg exporter until we move over fully to the new code.
- Set progress as soon as possible; previously the extracting and
gathering steps happened prior to the first progress.
- Add "extracting" and "gathering" progress steps, so that large imports
show more feedback in the early stage, and can be more quickly
interrupted.
* Add apkg export on backend
* Filter out missing media-paths at write time
* Make TagMatcher::new() infallible
* Gather export data instead of copying directly
* Revert changes to rslib/src/tags/
* Reuse filename_is_safe/check_filename_safe()
* Accept func to produce MediaIter in export_apkg()
* Only store file folder once in MediaIter
* Use temporary tables for gathering
export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.
* Use schedule_as_new() to reset cards
* ExportData → ExchangeData
* Ignore ascii case when filtering system tags
* search_notes_cards_into_table →
search_cards_of_notes_into_table
* Start on apkg importing on backend
* Fix due dates in days for apkg export
* Refactor import-export/package
- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.
* Add SafeMediaEntry for deserialized MediaEntries
* Prepare media based on checksums
- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.
* Handle encoding in `replace_media_refs()`
* Add trait to keep down cow boilerplate
* Add notetypes immediately instaed of preparing
* Move target_col into Context
* Add notes immediately instaed of preparing
* Note id, not guid of conflicting notes
* Add import_decks()
* decks_configs → deck_configs
* Add import_deck_configs()
* Add import_cards(), import_revlog()
* Use dyn instead of generic for media_fn
Otherwise, would have to pass None with type annotation in the default
case.
* Fix signature of import_apkg()
* Fix search_cards_of_notes_into_table()
* Test new functions in text.rs
* Add roundtrip test for apkg (stub)
* Keep source id of imported cards (or skip)
* Keep source ids of imported revlog (or skip)
* Try to keep source ids of imported notes
* Make adding notetype with id undoable
* Wrap apkg import in transaction
* Keep source ids of imported deck configs (or skip)
* Handle card due dates and original due/did
* Fix importing cards/revlog
Card ids are manually uniquified.
* Factor out card importing
* Refactor card and revlog importing
* Factor out card importing
Also handle missing parents .
* Factor out note importing
* Factor out media importing
* Maybe upgrade scheduler of apkg
* Fix parent deck gathering
* Unconditionally import static media
* Fix deck importing edge cases
Test those edge cases, and add some global test helpers.
* Test note importing
* Let import_apkg() take a progress func
* Expand roundtrip apkg test
* Use fat pointer to avoid propogating generics
* Fix progress_fn type
* Expose apkg export/import on backend
* Return note log when importing apkg
* Fix archived collection name on apkg import
* Add CollectionOpWithBackendProgress
* Fix wrong Interrupted Exception being checked
* Add ClosedCollectionOp
* Add note ids to log and strip HTML
* Update progress when checking incoming media too
* Conditionally enable new importing in GUI
* Fix all_checksums() for media import
Entries of deleted files are nulled, not removed.
* Make apkg exporting on backend abortable
* Return number of notes imported from apkg
* Fix exception printing for QueryOp as well
* Add QueryOpWithBackendProgress
Also support backend exporting progress.
* Expose new apkg and colpkg exporting
* Open transaction in insert_data()
Was slowing down exporting by several orders of magnitude.
* Handle zstd-compressed apkg
* Add legacy arg to ExportAnkiPackage
Currently not exposed on the frontend
* Remove unused import in proto file
* Add symlink for typechecking of import_export_pb2
* Avoid kwargs in pb message creation, so typechecking is not lost
Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.
* Avoid re-exporting protobuf msgs we only use internally
* Stop after one test failure
mypy often fails much faster than pylint
* Avoid an extra allocation when extracting media checksums
* Update progress after prepare_media() finishes
Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.
* Show progress of note imports
Note import is the slowest part, so showing progress here makes the UI
feel more responsive.
* Reset filtered decks at import time
Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.
We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).
https://github.com/ankitects/anki/pull/1743/files#r839346423
* Fix a corner-case where due dates were shifted by a day
This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.
* Log conflicting note in remapped nt case
* take_fields() → into_fields()
* Alias `[u8; 20]` with `Sha1Hash`
* Truncate logged fields
* Rework apkg note import tests
- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)
* Fix sorting of imported decks
Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.
* target[_id]s → existing_card[_id]s
* export_collection_extracting_media() → ...
export_into_collection_file()
* target_already_exists→card_ordinal_already_exists
* Add search_cards_of_notes_into_table.sql
* Imrove type of apkg export selector/limit
* Remove redundant call to mod_schema()
* Parent tooltips to mw
* Fix a crash when truncating note text
String::truncate() is a bit of a footgun, and I've hit this before
too :-)
* Remove ExportLimit in favour of separate classes
* Remove OpWithBackendProgress and ClosedCollectionOp
Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.
Also fix aborting of colpkg exports, which slipped through in #1817.
* Tidy up import log
* Avoid QDialog.exec()
* Default to excluding scheuling for deck list deck
* Use IncrementalProgress in whole import_export code
* Compare checksums when importing colpkgs
* Avoid registering changes if hashes are not needed
* ImportProgress::Collection → ImportProgress::File
* Make downgrading apkgs depend on meta version
* Generalise IncrementableProgress
And use it in entire import_export code instead.
* Fix type complexity lint
* Take count_map for IncrementableProgress::get_inner
* Replace import/export env with Shift click
* Accept all args from update() for backend progress
* Pass fields of ProgressUpdate explicitly
* Move update_interval into IncrementableProgress
* Outsource incrementing into Incrementor
* Mutate ProgressUpdate in progress_update callback
* Switch import/export legacy toggle to profile setting
Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.
One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.
* Show extension in export dialog
* Continue to provide separate options for schema 11+18 colpkg export
* Default to colpkg export when using File>Export
* Improve appearance of combo boxes when switching between apkg/colpkg
+ Deal with long deck names
* Convert newlines to spaces when showing fields from import
Ensures each imported note appears on a separate line
* Don't separate total note count from the other summary lines
This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.
* Fix 'deck not normal' error when importing a filtered deck for the 2nd time
* Fix [Identical] being shown on first import
* Revert "Continue to provide separate options for schema 11+18 colpkg export"
This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.
Will use a different approach
* Move legacy support into a separate exporter option; add to apkg export
* Adjust 'too new' message to also apply to .apkg import case
* Show a better message when attempting to import new apkg into old code
Previously the user could end seeing a message like:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte
Unfortunately we can't retroactively fix this for older clients.
* Hide legacy support option in older exporting screen
* Reflect change from paths to fnames in type & name
* Make imported decks normal at once
Then skip special casing in update_deck(). Also skip updating
description if new one is empty.
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* First attempt at adding a directory for icons under //ts
* Fix image import
* Fix import order
* Add cloze button group
* Fix issue with toolbar.toolbar dynamically slottable
* Change tooltip for repeating cloze deletion
* Fix repeat cloze button not working on macOS (dae)
* Collection needs to be closed prior to backup even when not downgrading
* Backups -> BackupLimits
* Some improvements to backup_task
- backup_inner now returns the error instead of logging it, so that
the frontend can discover the issue when they await a backup (or create
another one)
- start_backup() was acquiring backup_task twice, and if another thread
started a backup between the two locks, the task could have been accidentally
overwritten without awaiting it
* Backups no longer require a collection close
- Instead of closing the collection, we ensure there is no active
transaction, and flush the WAL to disk. This means the undo history
is no longer lost on backup, which will be particularly useful if we
add a periodic backup in the future.
- Because a close is no longer required, backups are now achieved with
a separate command, instead of being included in CloseCollection().
- Full sync no longer requires an extra close+reopen step, and we now
wait for the backup to complete before proceeding.
- Create a backup before 'check db'
* Add File>Create Backup
https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157
* Defer checkpoint until we know we need it
When running periodic backups on a timer, we don't want to be fsync()ing
unnecessarily.
* Skip backup if modification time has not changed
We don't want the user leaving Anki open overnight, and coming back
to lots of identical backups.
* Periodic backups
Creates an automatic backup every 30 minutes if the collection has been
modified.
If there's a legacy checkpoint active, tries again 5 minutes later.
* Switch to a user-configurable backup duration
CreateBackup() now uses a simple force argument to determine whether
the user's limits should be respected or not, and only potentially
destructive ops (full download, check DB) override the user's configured
limit.
I considered having a separate limit for collection close and automatic
backups (eg keeping the previous 5 minute limit for collection close),
but that had two downsides:
- When the user closes their collection at the end of the day, they'd
get a recent backup. When they open the collection the next day, it
would get backed up again within 5 minutes, even though not much had
changed.
- Multiple limits are harder to communicate to users in the UI
Some remaining decisions I wasn't 100% sure about:
- If force is true but the collection has not been modified, the backup
will be skipped. If the user manually deleted their backups without
closing Anki, they wouldn't get a new one if the mtime hadn't changed.
- Force takes preference over the configured backup interval - should
we be ignored the user here, or take no backups at all?
Did a sneaky edit of the existing ftl string, as it hasn't been live
long.
* Move maybe_backup() into Collection
* Use a single method for manual and periodic backups
When manually creating a backup via the File menu, we no longer make
the user wait until the backup completes. As we continue waiting for
the backup in the background, if any errors occur, the user will get
notified about it fairly quickly.
* Show message to user if backup was skipped due to no changes
+ Don't incorrectly assert a backup will be created on force
* Add "automatic" to description
* Ensure we backup prior to importing colpkg if collection open
The backup doesn't happen when invoked from 'open backup' in the profile
screen, which matches Anki's previous behaviour. The user could
potentially clobber up to 30 minutes of their work if they exited to
the profile screen and restored a backup, but the alternative is we
create backups every time a backup is restored, which may happen a number
of times if the user is trying various ones. Or we could go back to a
separate throttle amount for this case, at the cost of more complexity.
* Remove the 0 special case on backup interval; minimum of 5 minutes
https://github.com/ankitects/anki/pull/1728#discussion_r830876833
The old Python code was only checking for NFC encoding, but we should
check for other issues like special filenames on windows (eg con.mp3)
- On export, the user is told to use Check Media if their media has
invalid filenames.
- On import, legacy packages will be transparently normalized. Since we're
doing the checks on export as well, any invalid names in a v3 package
are an error.
* Implement colpkg exporting on backend
* Use exporting logic in backup.rs
* Refactor exporting.rs
* Add backend function to export collection
* Refactor backend/collection.rs
* Use backend for colpkg exporting
* Don't use default zip compression for media
* Add exporting progress
* Refactor media file writing
* Write dummy collections
* Localize dummy collection note
* Minimize dummy db size
* Use `NamedTempFile::new()` instead of `new_in`
* Drop redundant v2 dummy collection
* COLLECTION_VERSION -> PACKAGE_VERSION
* Split `lock_collection()` into two to drop flag
* Expose new colpkg in GUI
* Improve dummy collection message
* Please type checker
* importing-colpkg-too-new -> exporting-...
* Compress the media map in the v3 package (dae)
On collections with lots of media, it can grow into megabytes.
Also return an error in extract_media_file_names(), instead of masking
it as an optional.
* Store media map as a vector in the v3 package (dae)
This compresses better (eg 280kb original, 100kb hashmap, 42kb vec)
In the colpkg import case we don't need random access. When importing
an apkg, we will need to be able to fetch file data for a given media
filename, but the existing map doesn't help us there, as we need
filename->index, not index->filename.
* Ensure folders in the media dir don't break the file mapping (dae)
* Extend maximum answer time...
Previously the time allowed to answer a question was capped at 10 minutes.
While this makes sense for fact recall, it limits the utility of Anki when
used for solving problems that can take more time to work through. This
extends the maximum answer time to 2 hours, which seems to be a reasonable
upper limit for solving a math or algorithm question.
* Add warning when max answer time exceeds 10 minutes
* Move warning below input field
* Start using WithFloating for SelectedTagBadge
* Adjust arrow on WithFloating for all directions
* Move TagOptionsBadge to its own sub directory
* Show autocomplete menu via WithFloating
* Have WithFloating return asReference instead of initializing its own reference element
* Add html: overflow: hidden for editor
* Replace ButtonToolbar with generic div
* Move scroll logic into autocomplete item + restrict Popover width to 95vw
* Fix autocomplete menu after pressing enter after selecting
- should not trigger an autocomplete choose
* Overlap TagInput perfectly with Tag
* Satisfy formatter
* Fix autocompletion item scrolling too much
* Remove unused Tag.svelte focusable prop
* Remove console.log
* Fix floating arrow is a diamond in dark mode
* Set autocompletion menu to 80vw
Ideally this would have been in beta 6 :-) No add-ons appear to be
using customstudy.py/taglimit.py though, so it should hopefully not be
disruptive.
In the earlier custom study changes, we didn't get around to addressing
issue #1136. Now instead of trying to determine the maximum increase
to allow (which doesn't work correctly with nested decks), we just
present the total available to the user again, and let them decide. There's
plenty of room for improvement here still, but further work here might
be better done once we look into decoupling deck limits from deck presets.
Tags and available cards are fetched prior to showing the dialog now,
and will show a progress dialog if things take a while.
Tags are stored in an aux var now, so they don't inflate the deck
object size.
* Add forget prompt with options
- Restore original position
- Reset reps and lapses
* Restore position when resetting for export
* Add config context to avoid passing keys
* Add routine to fetch defaults; use method-specific enum (dae)
* Keep original position by default (dae)
* Fix code completion for forget dialog (dae)
Needs to be a symbolic link to the generated file
* Add zstd dep
* Implement backend backup with zstd
* Implement backup thinning
* Write backup meta
* Use new file ending anki21b
* Asynchronously backup on collection close in Rust
* Revert "Add zstd dep"
This reverts commit 3fcb2141d2be15f907269d13275c41971431385c.
* Add zstd again
* Take backup col path from col struct
* Fix formatting
* Implement backup restoring on backend
* Normalize restored media file names
* Refactor `extract_legacy_data()`
A bit cumbersome due to borrowing rules.
* Refactor
* Make thinning calendar-based and gradual
* Consider last kept backups of previous stages
* Import full apkgs and colpkgs with backend
* Expose new backup settings
* Test `BackupThinner` and make it deterministic
* Mark backup_path when closing optional
* Delete leaky timer
* Add progress updates for restoring media
* Write restored collection to tempfile first
* Do collection compression in the background thread
This has us currently storing an uncompressed and compressed copy of
the collection in memory (not ideal), but means the collection can be
closed without waiting for compression to complete. On a large collection,
this takes a close and reopen from about 0.55s to about 0.07s. The old
backup code for comparison: about 0.35s for compression off, about
8.5s for zip compression.
* Use multithreading in zstd compression
On my system, this reduces the compression time of a large collection
from about 0.55s to 0.08s.
* Stream compressed collection data into zip file
* Tweak backup explanation
+ Fix incorrect tab order for ignore accents option
* Decouple restoring backup and full import
In the first case, no profile is opened, unless the new collection
succeeds to load.
In the second case, either the old collection is reloaded or the new one
is loaded.
* Fix number gap in Progress message
* Don't revert backup when media fails but report it
* Tweak error flow
* Remove native BackupLimits enum
* Fix type annotation
* Add thinning test for whole year
* Satisfy linter
* Await async backup to finish
* Move restart disclaimer out of backup tab
Should be visible regardless of the current tab.
* Write restored collection in chunks
* Refactor
* Write media in chunks and refactor
* Log error if removing file fails
* join_backup_task -> await_backup_completion
* Refactor backup.rs
* Refactor backup meta and collection extraction
* Fix wrong error being returned
* Call sync_all() on new collection
* Add ImportError
* Store logger in Backend, instead of creating one on demand
init_backend() accepts a Logger rather than a log file, to allow other
callers to customize the logger if they wish.
In the future we may want to explore using the tracing crate as an
alternative; it's a bit more ergonomic, as a logger doesn't need to be
passed around, and it plays more nicely with async code.
* Sync file contents prior to rename; sync folder after rename.
* Limit backup creation to once per 30 min
* Use zstd::stream::copy_decode
* Make importing abortable
* Don't revert if backup media is aborted
* Set throttle implicitly
* Change force flag to minimum_backup_interval
* Don't attempt to open folders on Windows
* Join last backup thread before starting new one
Also refactor.
* Disable auto sync and backup when restoring again
* Force backup on full download
* Include the reason why a media file import failed, and the file path
- Introduce a FileIoError that contains a string representation of
the underlying I/O error, and an associated path. There are a few
places in the code where we're currently manually including the filename
in a custom error message, and this is a step towards a more consistent
approach (but we may be better served with a more general approach in
the future similar to Anyhow's .context())
- Move the error message into importing.ftl, as it's a bit neater
when error messages live in the same file as the rest of the messages
associated with some functionality.
* Fix importing of media files
* Minor wording tweaks
* Save an allocation
I18n strings with replacements are already strings, so we can skip the
extra allocation. Not that it matters here at all.
* Terminate import if file missing from archive
If a third-party tool is creating invalid archives, the user should know
about it. This should be rare, so I did not attempt to make it
translatable.
* Skip multithreaded compression on small collections
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* Clarify some comments
* Don't destructure insertion trigger
* Make superscript and subscript use domlib/surround
* Create new {Text,Highlight}ColorButton
* Use domlib/surround for textcolor
- However there's still a crucial bug, when you're breaking existing
colored span when unsurrounding, their color is not restored
* Add underline format to removeFormats
* Simplify type of ElementMatcher and ElementClearer for end users
* Add some comments for normalize-insertion-ranges
* Split normalize-insertion-ranges into remove-adjacent and remove-within
* Factor out find-remove from unsurround.ts
* Rename merge-mach, simplify remove-within
* Clarify some comments
* Refactor first reduce
* Refactor reduceRight
* Flatten functions in merge-ranges
* Move some functionality to merge-ranges and do not export
* Refactor merge-ranges
* Remove createInitialMergeMatch
* Finish refactoring of merge-ranges
* Refactor merge-ranges to minimal-ranges and add some unit testing
* Move more logic into text-node
* Remove most most of the logic from remove-adjacent
- remove-adjacent is still part of the "merging" logic, as it increases
the scope of the child node ranges
* Add some tests for edge cases
* Merge remove-adjacent logic into minimal-ranges
* Refactor unnecessary list destructuring
* Add some TODOs
* Put removing nodes and adding new nodes into sequence
* Refactor MatchResult to MatchType and return clear from matcher
* Inline surround/helpers
* Shorten name of param
* Add another edge case test
* Add an example where commonAncestorContainer != normalization level
* Fix bug in find-adjacent when find more than one nested nodes
* Allow comments for Along type
* Simplify find-adjacent by removing intermediate and/or curried functions
* Remove extend-adjacent
* Add more tests when find-adjacent finds by descension
* Fix find-adjacent descending into block-level elements
* Add clarifying comment to refusing to descend into block-level elements
* Move shifting logic into find-adjacent
* Rename file matcher to match-type
* Give a first implemention of TreeVertex
* Remove MatchType.ALONG
- findAdjacent now directly modifies the range
* Rename MatchType.MATCH into MatchType.REMOVE
* Implement a version of find-within that utilizies match-tree
* Turn child node range into a class
* Fix bug in new find-adjacent function
* Make all find-adjacent tests test for ranges
* Surrounding within farthestMatchingAncestor when available
* Fix an issue with negligable elements
- also rename "along" elements to "negligable"
* Add two TODOs to SurroundFormat interface
* Have a messy first implementation of the new tree-node algorithm
* Maintain whether formatting nodes are covered or within user selection
* Move covered and insideRange into TreeNode superclass
* Reimplement findAdjacent logic
* Add extension logic
* Add an evaluate method to nodes
* Introduce BlockNode
* Add a first evaluate implementation
* Add left shift and inner shift logic
* Implement SurroundFormatUser
* Allow pass in formatter, ascender and merger from outside
* Fix insideRange and covered switch-up
* Fix MatchNode.prototype.isAscendable
* Fix another switch-up of covered and insideRange...
* Remove a lot of old code
* Have surround functions only return the range
- I still cannot think of a good reason why we should return addedNodes
and removedNodes, except for testing.
* Create formatting-tree directory
* Create build-tree directory + Move find-above up to /domlib
* Remove range-anchors
* Move unsurround logic into no-splitting
* Fix extend-merge
* Fix inner shift being eroneusly returned as left shift
* Fix oversight in SplitRange
* Redefine how ranges are recreated
* Rename covered to insideMatch and put as fourth parameter instead of third
* Keep track of match holes and match leaves
* Rename ChildNodeRange to FlatRange
* Change signature of matcher
* Fix bug in extend-merge
* Improve Match class
* Utilize cache in TextColorButton
* Implement getBaseSurrounder for TextColorButton
* Add matchAncestors field to FormattingNode
* Introduce matchAncestors and getCache
* Do clearing during parsing already
- This way, you know whether elements will be removed before getting to
Formatting nodes
* Make HighlightColorButton use our surround mechanism
* Fix a bug with calling .removeAttribute and .hasAttribute
* Add side button to RemoveFormat button
* Add disabled to remove format side button
* Expose remove formats on RemoveFormat button
* Reinvent editor/surround as Surrounder class
* Fix split-text when working with insert trigger
* Try counteracting the contenteditable's auto surrounding
* Remove matching elements before normalizing
* Rewrite match-type
* Move setting match leaves into build
* Change editing strings
- So that color strings match bold/italic strings better
* Fix border radius of List options menu
* Implement extensions functionality
* Remove some unnecessary code
* Fix split range endOffset
* Type MatchType
* Reformat MatchType + add docs
* Fix domlib/surround/apply
* Satisfy last tests
* Register Surrounder as package
* Clarify some comments
* Correctly implement reformat
* Reformat with inactive eraser formats
* Clear empty spans with RemoveFormatButton
* Fix Super/Subscript button
* Use ftl string for hardcoded tooltip
* Adjust wording
* Add main view menu
* Add browser view menu
* Use standard keys for zooming and full screen
* Capitalise menu item names
* Toggle Showing Cards/Notes -> Toggle Cards/Notes
* Explicitly set linux full screen key
on_toggle_fullscreen -> on_toggle_full_screen
* avoid repinning Rust deps by default
* add id_tree dependency
* Respect intermediate child limits in v3
* Test new behaviour of v3 counts
* Rework v3 queue building to respect parent limits
* Add missing did field to SQL query
* Fix `LimitTreeMap::is_exhausted()`
* Rework tree building logic
https://github.com/ankitects/anki/pull/1638#discussion_r798328734
* Add timer for build_queues()
* `is_exhausted()` -> `limit_reached()`
* Move context and limits into `QueueBuilder`
This allows for moving more logic into QueueBuilder, so less passing
around of arguments. Unfortunately, some tests will require additional
work to set up.
* Fix stop condition in new_cards_by_position
* Fix order gather order of new cards by deck
* Add scheduler/queue/builder/burying.rs
* Fix bad tree due to unsorted child decks
* Fix comment
* Fix `cap_new_to_review_rec()`
* Add test for new card gathering
* Always sort `child_decks()`
* Fix deck removal in `cap_new_to_review_rec()`
* Fix sibling ordering in new card gathering
* Remove limits for deck total count with children
* Add random gather order
* Remove bad sibling order handling
All routines ensure ascending order now.
Also do some other minor refactoring.
* Remove queue truncating
All routines stop now as soon as the root limit is reached.
* Move deck fetching into `QueueBuilder::new()`
* Rework new card gather and sort options
https://github.com/ankitects/anki/pull/1638#issuecomment-1032173013
* Disable new sort order choices ...
depending on set gather order.
* Use enum instead of numbers
* Ensure valid sort order setting
* Update new gather and sort order tooltips
* Warn about random insertion order with v3
* Revert "Add timer for build_queues()"
This reverts commit c9f5fc6ebe87953c17a0c842990b009b5596c69c.
* Update rslib/src/storage/card/mod.rs (dae)
* minor wording tweaks to the tooltips (dae)
+ move legacy strings to bottom
+ consistent capitalization (our leech action still needs fixing,
but that will require introducing a new 'suspend card' string as the
existing one is used elsewhere as well)
* Move some AddCards specific code to NoteCreator.svelte
* Add new strings for Toggling the Visual / HTML editor
* Set LabelContainer vertical-align to text-top
- Makes them look more centered
* Remove appendInParentheses helper
* Make all ts/*.html files include only module.js and module.css
* Move any JS from .html to index files
* Remove .html files from ts modules
* Remove Python with Starlark implemenation
* Remove reference to non-existing file
* Remove deck-option.html as well
* fix change-notetype screen (dae)
* Add new `card_rendering` mod
Parses a text with av/tts tags and strips or extracts tags.
* Replace old `extract_av_tags` and `strip_av_tags`
... with new `card_rendering` mod
* ressource -> resource
* Add AV prettifier for use in browser table
* Accept String in av tag routines
... and avoid redundant writes if no changes need to be made.
* add benchmarking with criterion; make links test optional (dae)
cargo install cargo-criterion, then run ./bench.sh
* performance comparison: creating HashMap up front (dae)
the previous solution:
anki_tag_parse time: [1.8401 us 1.8437 us 1.8476 us]
this solution:
anki_tag_parse time: [2.2420 us 2.2447 us 2.2477 us]
change: [+21.477% +21.770% +22.066%] (p = 0.00 < 0.05)
Performance has regressed.
* Revert "performance comparison: creating HashMap up front" (dae)
This reverts commit f19126a2f15b729b825825a49283f63ab13474d0.
* add missing header
* Write error message if tts lang is missing
* `Tag` -> `Directive`
* Allow theme change at runtime and add hook
* Save or restore default palette on theme change
* Update aqt widget styles on theme change
* styling fixes
- drop _light_palette, as default_palette serves the same purpose
- save default platform theme, and restore it when switching away
from nightmode
- update macOS light/dark mode on theme switch
- fix unreadable menus on Windows
* update night-mode classes on theme change
This is the easy part - CSS styling that uses standard_css or our
css variables should update automatically. The main remaining issue
is JS code that sets colors based on the theme at the time it's run -
eg the graph code, and the editor.
* switch night mode value on toggle
* expose current theme via a store; switch graphs to use it
https://github.com/ankitects/anki/issues/1471#issuecomment-972402492
* start using currentTheme in editor/components
This fixes basic editing - there are still components that need updating.
* add simple xcodeproj for code completion
* add helper to get currently-active system theme on macOS
* fix setCurrentTheme not being immediately available
* live update tag color
* style().name() doesn't work on Qt5
* automatic theme switching on Windows/Mac
* currentTheme -> pageTheme
* Replace `nightModeKey` with `pageTheme`
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* Enable access to old notetype name
* Set minimum height for ChangeNotetypeDialog
* Add bootstrap icons to change-notetype
* Move alert up and make it collapsible
* Tweak some CSS
- Add variables --sticky-bg and --sticky-border to StickyContainer
- Tweak base.css
* Add translatable string "(Nothing)"
* Rework ChangeNotetype screen
* Initially load option at newIndex and remaining options on focus
Optimization for big notetypes:
Should increase efficiency from O(n²) to O(n). Test on notetype with 500 templates shows significant improvement in load time (~10s down to ~1s).
* Try to satisfy rust test
* Change arrow direction depending on reading direction
+ add 0.5em top padding to main
* Create Alert.svelte
* Introduce CSS variable --pane-bg
* Revert "Initially load option at newIndex and remaining options on focus"
This reverts commit f42beee45c27dba9433d76217fb583b117fb5231.
* Final cleanup
* Refine padding/gutter
* Remove unnecessary stopPropagation of mathjax-overlay events
* Use CodeMirror component for MathjaxHandle
* Refactor ResizeObserver code in MathjaxHandle
* Wrap setRange in CodeMirror in try/catch
* Add Mathjax Editor bottom margin
* Add custom Enter and Shift+Enter shortcuts for the MathjaxHandle
* Format
* Move placeCaretAfter to domlib
* Move focus back to field after editing Mathjax
* Put Cursor after Mathjax after accepting
* Add delete button for Mathjax
* Change border color of mathjax menu
* Refactor into MathjaxMenu
* Put caretKeyword in variable
* Use one ResizeObserver for all Mathjax images
* Add minmimum width for Mathjax editor
* is still smaller than minimal window width
* Add bazel directories to .prettierignore and format from root
* exclude ftl/usage (dae)
the json files that live there are output from our tooling, and
formatting them means an extra step each time we want to update them
also exclude .mypy_cache, which is output by scripts/mypy*
* minor ftl tweak: newline -> new line (dae)
* Canonify import of i18n module
Should always be imported as `tr`, or `tr2` if there is a name collision
(Svelte).
* Add helper for garbage collecting ftl strings
Also add a serializer for ftl asts.
* Add helper for filter-mapping `DirEntry`s
* Fix `i18n_helpers/BUILD.bazel`
* run cargo-raze
* Refactor `garbage_collection.rs`
- Improve helper for file iterating
- Remove unused terms as well
- Fix issue with checking for nested messages by switching to a regex-
based approach (which runs before deleting)
- Some more refactorings and lint fixes
* Fix lints in `serialize.rs`
* Write json pretty and sorted
* Update `serialize.rs` and fix header
* Fix doc and remove `dbg!`
* Add binaries for ftl garbage collection
Also relax type constraints and strip debug tests.
* add rust_binary targets for i18n helpers (dae)
* add scripts to update desktop usage/garbage collect (dae)
Since we've already diverged from 2.1.49, we won't gain anything
from generating a stable json just yet. But once 2.1.50 is released,
we should run 'ftl/update-desktop-usage.sh stable'.
* add keys from AnkiMobile (dae)
* Mention caveats in `remove-unused.sh`
* Add description input to fields dialog
QLineEdit seems like the best option, as it saves space and motivates users to keep their descriptions concise.
* Add setDescriptions to note initialization script
Went for the extra function instead of including it in setFields to prevent potential add-on breakages.
* Add tooltip next to field name if description is set
* Refactor code according to suggestions
Set default tooltip placement to right instead of bottom
Use .get() for fld["description"]
Fix tab order in fields dialog
Swap out abbreviation "desc" for full length name to keep consistency
* Update Protobuf and Rust for description
Add description to notetypes.proto and schema11
Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>
* Fix tooltips not updating with description
Remove redundant variable tooltipOptions
Update previousTooltip within reactive function
* Move LabelDescription out of LabelName
Co-authored-by: Henrik Giesel <hgiesel@users.noreply.github.com>
* Decrease icon size and fix alignment
Co-Authored-By: Henrik Giesel <hengiesel@gmail.com>
* the new key needs to be cleared from fields, not the notetype itself
Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>
Co-authored-by: Henrik Giesel <hengiesel@gmail.com>
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
- changes can now be undone
- the same field can now be mapped to multiple target fields, allowing
fields to be cloned
- the old Qt dialog has been removed
- the old col.models.change() API calls the new code, to avoid
breaking existing consumers. It requires the field map to always
be passed in, but that appears to have been the common case.
- closes#1175
The hard limit from sqlite may be larger, but things slow down as more
tags are selected.
https://forums.ankiweb.net/t/unable-to-create-custom-test/10467
There are a number of things that could be improved here:
- we should show a live count so users are aware of the limit
- we should be filling in the parent tags when they're not explicitly
listed on a card
- we should reconsider disabling the 'tags to include' by default
It may make sense to defer these changes until we can move this screen
into Svelte/handle the processing in the backend.
Combine existing check for unparsable templates with a check for unknown
field names and a check for front sides without any field replacement.
Updating the notetype's fields now mutates the parsed templates, so the
checks can run on the final templates.
Also:
- fix issues where the Undo action in the Browse screen was not
consistent with the main window. The existing hook signature has been
changed; from a snapshot of the add-on code from a few months ago, it
was not a hook that was being used by anyone.
- change the undo shortcut in the Browse window to match the main
window. It was different because undoing a change in the editing area
could accidentally trigger an undo of an operation, but the damage is
limited now that (most) operations can be redone. If it still proves to
be a problem, perhaps we should just always swallow ctrl+z when an
editing field is focused.
- Daily limits are no longer inherited - each deck limits its own
cards, and the selected deck enforces a maximum limit.
- Fetching of review cards now uses a single query, and sorts in advance.
In collections with a large number of overdue cards and decks, this is
faster than iterating over each deck in turn.
- Include interday learning count in review count & review limit, and
allow them to be buried.
- Warn when parent review limit is lower than child deck in deck options.
- Cap the new card limit to the review limit.
- Add option to control whether new card fetching short-circuits.
Updating a deck via protobuf is now exposed on the backend, but not
currently on the frontend - I suspect we'll be better off writing
separate routines for the actions we need instead, and we get a better
undo description for free.
This is currently causing an ugly redraw in the browse screen, which
will need fixing.
Older translations will note have the $notetype variable, but that is
not an error in Fluent - it would only cause problems if we tried to
use the new string on older Anki versions.
- Filtered deck creation now happens as an atomic operation, and is
undoable.
- The logic for initial search text, normalizing searches and so on
has been pushed into the backend.
- Use protobuf to pass the filtered deck to the updated dialog, so
we don't need to deal with untyped JSON.
- Change the "revise your search?" prompt to be a simple info box -
user has access to cancel and build buttons, and doesn't need a separate
prompt. Tweak the wording so the 'show excluded' button should be more
obvious.
- Filtered decks have a time appended to them instead of a number,
primarily because it's easier to implement. No objections going back to
the old behaviour if someone wants to contribute a clean patch.
The standard de-duplication will happen if two decks are created in the
same minute with the same name.
- Tweak the default sort order, and start with two searches. The UI
will still hide the second search by default, but by starting with two,
the frontend doesn't need logic for creating the starting text.
- Search errors now have their own error type, instead of using
InvalidInput, as that was intended mainly for bad API calls. The markdown
conversion is done when the error is converted from the backend, allowing
errors to printed as a string without any special handling by the calling
code.
TODO: when building a new filtered deck, update_active() is clobbering
the undo log when the overview is refreshed
- clear_unused_tags() is now undoable, and returns the number of removed
notes
- add a new mw.query_op() helper for immutable queries
- decouple "freeze/unfreeze ui state" hooks from the "interface update
required" hook, so that the former is fired even on error, and can be
made re-entrant
- use a 'block_updates' flag in Python, instead of setUpdatesEnabled(),
as the latter has the side-effect of preventing child windows like
tooltips from appearing, and forces a full redrawn when updates are
enabled again. The new behaviour leads to the card list blanking out
when a long-running op is running, but in the future if we cache the
cell values we can just display them from the cache instead.
- we were indiscriminately saving the note with saveNow(), due to the
call to saveTags(). Changed so that it only saves when the tags field
is focused.
- drain the "on_done" queue on main before launching a new background
task, to lower the chances of something in on_done making a small query
to the DB and hanging until a long op finishes
- the duplicate check in the editor was executed after the webview loads,
leading to it hanging until the sidebar finishes loading. Run it at
set_note() time instead, so that the editor loads first.
- don't throw an error when a long-running op started with with_progress()
finishes after the window it was launched from has closed
- don't throw an error when the browser is closed before the sidebar
has finished loading
- This avoids the need for a separate screen, though we may want to
slightly fade out the display when information is stale.
- Means the browser can delay updates just like the main window does.
'card modified' covers the common case where we need to rebuild the
study queue, but is also set when changing the card flags. We want to
avoid a queue rebuild in that case, as it causes UI flicker, and may
result in a different card being shown. Note marking doesn't trigger
a queue build, but still causes flicker, and may return the user back
to the front side when they were looking at the answer.
I still think entity-based change tracking is the simplest in the
common case, but to solve the above, I've introduced an enum describing
the last operation that was taken. This currently is not trying to list
out all possible operations, and just describes the ones we want to
special-case.
Other changes:
- Fire the old 'state_did_reset' hook after an operation is performed,
so legacy code can refresh itself after an operation is performed.
- Fire the new `operation_did_execute` hook when mw.reset() is called,
so that as the UI is updated to the use the new hook, it will still
be able to refresh after legacy code calls mw.reset()
- Update the deck browser, overview and review screens to listen to
the new hook, instead of relying on the main window to call moveToState()
- Add a 'set flag' backend action, so we can distinguish it from a
normal card update.
- Drop the separate added/modified entries in the change list in
favour of a single entry per entity.
- Add typing to mw.state
- Tweak perform_op()
- Convert a few more actions to use perform_op()
Up until now, we've been forcing a new search whenever reset is called.
The primary reason was that the card list display routines did not expect
a card or note to have been removed. By updating the model to show
"(deleted)" when a card or note is missing, we no longer have to repeat
the search.
This has a few advantages:
- Searches, especially complex ones, can be slow to execute. When we
perform them after every operation like a delete, it can make Anki feel
sluggish.
- The fact that notes have been deleted becomes more obvious - some users
found it easy to miss the "deleted" pop-up in the past.
This change does not just affect deletions, as many other operations
trigger a reset as well. In the past, when using 'set due date' in the
review screen for example, it caused an ugly flicker in the browser screen,
and could be slow when the current search couldn't be quickly redone.
The disadvantage of this approach is that the displayed content may
not reflect the specified search, which has the potential to be confusing.
But if that turns out to be a problem, it could be (partly) alleviated by
displaying a refresh button next to the search bar when the search may
need to be refreshed.
Feedback welcome!
Work in progress - still to do:
- renames appear as 'Update Deck' - easiest way to solve it would
be to have a separate backend method for renames
- drag&drop of decks not yet undoable
- since the undo status is updated after the backend method ends,
the older checkpoint() calls need to be replaced with an
update_undo_status() at the end of the call - if we just remove the
checkpoint, then the menu doesn't get updated
This splits update_card() into separate undoable/non-undoable ops
like the change to notes in b4396b94abdeba3347d30025c5c0240d991006c9
It means that actions get a blanket 'Update Card' description - in the
future we'll probably want to either add specific actions to the backend,
or allow an enum or string to be passed in to describe the op.
Other changes:
- card.flush() can no longer be used to add new cards. Card creation
is only supposed to be done in response to changes in a note's fields,
and this functionality was only exposed because the card generation
hadn't been migrated to the backend at that point. As far as I'm aware,
only Arthur's "copy notes" add-on used this functionality, and that should
be an easy fix - when the new note is added, the associated cards will
be generated, and they can then be retrieved with note.cards()
- tidy ups/PEP8
- note.flush() behaves like before, as otherwise actions or add-ons
that perform bulk flushing would end up creating an undo entry for
each note
- added col.update_note() to opt in to the new behaviour
- tidy up the names of some related routines
