Commit Graph

1915 Commits

Author SHA1 Message Date
Matthias Metelka
d1ba48bc48
Expose cloze text as HTML attribute on question side (#1968)
* Expose cloze text as attribute on front side

* Update test_models.py

* Update template_filters.rs

* Escape HTML for data-attribute

* Use minimal HTML encoding in Rust

to match Python's html.escape and pass tests.

* Rename attribute to data-cloze

to make it more generic.

* Run formatter

* Revert to using Rust encode_attribute and add helper function for tests
2022-07-19 04:22:57 +10:00
RumovZ
f6390e9455
Adjust remaining steps after config update (#1956)
* Adjust remaining steps after config update

* Handle relearning steps separately

Also refactor a lot.

* Also adjust remaining steps after deck change

* Test step adjustment after config update

* Fix `SearchBuilder::(re)learning_cards()`

* Fix step adjustment after deck change

* Test step adjustment after deck change

* Fix test name

* Readjust remaining steps according to last delay

Also atomize tests and add some tooling.
2022-07-14 11:24:34 +10:00
RumovZ
8c515e316e
Template err improvements (#1953)
* Throw error for unknown condition fields as well

So if 'foo' is not a field, refuse to save a template containing
`{{#foo}}bar{{/foo}}`. Previously, only `{{foo}}` would be checked.
As a side effect, templates which *only* contain fields as conditions
may be saved. Meh.

* Display template errors in q/a columns only

So the affected browser row remains active and the user can fix the
template more easily.

* Specify if error occured in a browser template

* Minor wording tweak (dae)

There's an argument for using the exact wording as well, but this just
reads a little more naturally to me.
2022-07-09 13:00:03 +10:00
Sam Penny
9dcceff4af
Remember previous choices in reposition dialog (#1950)
* remember previous choices in reposition dialog

* remember previous choice for randomize option as well

* fix failing test
2022-07-08 11:28:38 +10:00
RumovZ
2bbd1fca2a
Skip card generation if fronts remain unchanged (#1949)
Closes #1945
2022-07-06 19:26:14 +10:00
RumovZ
bebc6fd23a
Disregard manual reschedulings in introduced:x (#1932) 2022-06-27 17:20:36 +10:00
Damien Elmes
5ea78e1c8e Since DupeResolution is in CsvMetadata, we don't need to pass it separately
Follow-up to #1930
2022-06-27 17:15:54 +10:00
Damien Elmes
67e4edcd8b Expose backend_proto publicly for AnkiDroid, and rename to pb
We were aliasing it on import half the time anyway
2022-06-27 15:27:53 +10:00
Damien Elmes
6ff159304d Fix backups failing on Android 2022-06-27 15:27:51 +10:00
RumovZ
b6a7760cb4
Restore and save last dupe resolution setting (#1930)
* Restore dupe resolution setting

* Save dupe resolution setting

* Push config logic into backend (dae)
2022-06-24 15:10:06 +10:00
RumovZ
8478492190
Check ids when gathering data (#1928)
This will throw an error if a card, note or revlog id from the future
is found during apkg import or export.
2022-06-24 13:56:52 +10:00
RumovZ
1b8bab73d1
Csv import tweaks (#1920)
* Don't use special label for tags column

This was setting the label of whichever column the user chose for tags
to "Tags". Showing field content is more helpful.

* Map tags column automatically like fields
2022-06-22 09:44:12 +10:00
Damien Elmes
4d51ee8a64 Fix number of days in a (standard) year
https://forums.ankiweb.net/t/prop-ivl-faulty/20665
2022-06-13 09:09:27 +10:00
Damien Elmes
f15e294981 Fix collection load failure on Android
Loading a larger collection with V16 enabled was failing in openCollection
with error 6410, similar to https://github.com/simolus3/drift/issues/876
2022-06-10 23:21:35 +10:00
RumovZ
6da5e5b042
CSV import/export fixes and features (#1898)
* Fix footer moving upwards

* Fix column detection

Was broken because escaped line breaks were not considered.
Also removes delimiter detection on `#columns:` line. User must use tabs
or set delimiter beforehand.

* Add CSV preview

* Parse `#tags column:`

* Optionally export deck and notetype with CSV

* Avoid clones in CSV export

* Prevent bottom of page appearing under footer (dae)

* Increase padding to 1em (dae)

With 0.5em, when a vertical scrollbar is shown, it sits right next to
the right edge of the content, making it look like there's no right
margin.

* Experimental changes to make table fit+scroll (dae)

- limit individual cells to 15em, and show ellipses when truncated
- limit total table width to body width, so that inner table is shown
with scrollbar
- use class rather than id - ids are bad practice in Svelte components,
as more than one may be displayed on a single page

* Skip importing foreign notes with filtered decks

Were implicitly imported into the default deck before.
Also some refactoring to fetch deck ids and names beforehand.

* Hide spacer below hidden field mapping

* Fix guid being replaced when updating note

* Fix dupe identity check

Canonify tags before checking if dupe is identical, but only add update
tags later if appropriate.

* Fix deck export for notes with missing card 1

* Fix note lines starting with `#`

csv crate doesn't support escaping a leading comment char. :(

* Support import/export of guids

* Strip HTML from preview rows

* Fix initially set deck if current is filtered

* Make isHtml toggle reactive

* Fix `html_to_text_line()` stripping sound names

* Tweak export option labels

* Switch to patched rust-csv fork

Fixes writing lines starting with `#`, so revert 5ece10ad05f331.

* List column options with first column field

* Fix flag for exports with HTML stripped
2022-06-09 10:28:01 +10:00
Damien Elmes
59ee399c5f Update regex crate for CVE 2022-06-07 08:39:07 +10:00
RumovZ
42cbe42f06
Plaintext import/export (#1850)
* Add crate csv

* Add start of csv importing on backend

* Add Menomosyne serializer

* Add csv and json importing on backend

* Add plaintext importing on frontend

* Add csv metadata extraction on backend

* Add csv importing with GUI

* Fix missing dfa file in build

Added compile_data_attr, then re-ran cargo/update.py.

* Don't use doubly buffered reader in csv

* Escape HTML entities if CSV is not HTML

Also use name 'is_html' consistently.

* Use decimal number as foreign ease (like '2.5')

* ForeignCard.ivl → ForeignCard.interval

* Only allow fixed set of CSV delimiters

* Map timestamp of ForeignCard to native due time

* Don't trim CSV records

* Document use of empty strings for defaults

* Avoid creating CardGenContexts for every note

This requires CardGenContext to be generic, so it works both with an
owned and borrowed notetype.

* Show all accepted file types  in import file picker

* Add import_json_file()

* factor → ease_factor

* delimter_from_value → delimiter_from_value

* Map columns to fields, not the other way around

* Fallback to current config for csv metadata

* Add start of new import csv screen

* Temporary fix for compilation issue on Linux/Mac

* Disable jest bazel action for import-csv

Jest fails with an error code if no tests are available, but this would
not be noticable on Windows as Jest is not run there.

* Fix field mapping issue

* Revert "Temporary fix for compilation issue on Linux/Mac"

This reverts commit 21f8a261408cdae49ec031aa21a1b659c4f66d82.

* Add HtmlSwitch and move Switch to components

* Fix spacing and make selectors consistent

* Fix shortcut tooltip

* Place import button at the top with path

* Fix meta column indices

* Remove NotetypeForString

* Fix queue and type of foreign cards

* Support different dupe resolution strategies

* Allow dupe resolution selection when importing CSV

* Test import of unnormalized text

Close  #1863.

* Fix logging of foreign notes

* Implement CSV exports

* Use db_scalar() in notes_table_len()

* Rework CSV metadata

- Notetypes and decks are either defined by a global id or by a column.
- If a notetype id is provided, its field map must also be specified.
- If a notetype column is provided, fields are now mapped by index
instead of name at import time. So the first non-meta column is used for
the first field of every note, regardless of notetype. This makes
importing easier and should improve compatiblity with files without a
notetype column.
- Ensure first field can be mapped to a column.
- Meta columns must be defined as `#[meta name]:[column index]` instead
of in the `#columns` tag.
- Column labels contain the raw names defined by the file and must be
prettified by the frontend.

* Adjust frontend to new backend column mapping

* Add force flags for is_html and delimiter

* Detect if CSV is HTML by field content

* Update dupe resolution labels

* Simplify selectors

* Fix coalescence of oneofs in TS

* Disable meta columns from selection

Plus a lot of refactoring.

* Make import button stick to the bottom

* Write delimiter and html flag into csv

* Refetch field map after notetype change

* Fix log labels for csv import

* Log notes whose deck/notetype was missing

* Fix hiding of empty log queues

* Implement adding tags to all notes of a csv

* Fix dupe resolution not being set in log

* Implement adding tags to updated notes of a csv

* Check first note field is not empty

* Temporary fix for build on Linux/Mac

* Fix inverted html check (dae)

* Remove unused ftl string

* Delimiter → Separator

* Remove commented-out line

* Don't accept .json files

* Tweak tag ftl strings

* Remove redundant blur call

* Strip sound and add spaces in csv export

* Export HTML by default

* Fix unset deck in Mnemosyne import

Also accept both numbers and strings for notetypes and decks in JSON.

* Make DupeResolution::Update the default

* Fix missing dot in extension

* Make column indices 1-based

* Remove StickContainer from TagEditor

Fixes line breaking, border and z index on ImportCsvPage.

* Assign different key combos to tag editors

* Log all updated duplicates

Add a log field for the true number of found notes.

* Show identical notes as skipped

* Split tag-editor into separate ts module (dae)

* Add progress for CSV export

* Add progress for text import

* Tidy-ups after tag-editor split (dae)

- import-csv no longer depends on editor
- remove some commented lines
2022-06-01 20:26:16 +10:00
dobefore
b4cffe2339
rust impl stdError trait (#1890)
* impl stdError trait

* Update CONTRIBUTORS
2022-05-31 17:52:12 +10:00
Damien Elmes
5d835ae813 Always generate missing cards when saving card templates
https://forums.ankiweb.net/t/anki-doesnt-generate-cards-after-un-doing-selective-card-generation/20283
2022-05-26 12:44:57 +10:00
Damien Elmes
82196753ec Rework display of available cards in custom study
In v3, it's more informative to show the count of child decks separately,
since increasing the limit of the current deck does not increase the limits
of child decks. When we rework the decks list in the future, a tooltip
will hopefully provide an easier way for users to see where cards are
available, and where limits are being applied.

Closes #1868
2022-05-20 17:52:02 +10:00
RumovZ
6c807c1b2e
Fix review queue if limit is reached (#1855) 2022-05-10 12:11:35 +10:00
Damien Elmes
d946e5ddd5
Fix for crash with invalid dates on Windows (#1837)
https://forums.ankiweb.net/t/bug-report-crashing-when-opening-deck-browser/19768

Caused by a note mtime that was 1000x larger than it should have been.
Check DB will now fix this case (but there are others it still does not
cover, such as invalid card/note IDs).

https://docs.rs/chrono/0.4.19/x86_64-pc-windows-msvc/src/chrono/sys/windows.rs.html#128
2022-05-07 10:30:23 +10:00
Damien Elmes
c8a4e5ea22 Add more progress + abort points to export code
The old `media_files_did_export` hook has been kept around for use with
the legacy apkg exporter (an add-on uses it), and a new 
`legacy_export_progress` hook has been added so we can get progress
from the new colpkg exporter until we move over fully to the new code.
2022-05-06 15:32:23 +10:00
RumovZ
99c4ec93d2
Fix some apkg import issues (#1836)
* Allow gathering of orphaned decks

* Skip redundant media checking in apkg import
2022-05-06 09:22:15 +10:00
Damien Elmes
ac1cbaf719 Fix implicit transaction not being cleared on transact() failure
"release foo" clears the implicit outer transaction, but "rollback to foo"
does not.
2022-05-05 13:39:41 +10:00
Damien Elmes
1297beff63 Minor progress tweaks to apkg importing
- Set progress as soon as possible; previously the extracting and
gathering steps happened prior to the first progress.
- Add "extracting" and "gathering" progress steps, so that large imports
show more feedback in the early stage, and can be more quickly
interrupted.
2022-05-05 12:23:05 +10:00
RumovZ
5f9451f547
Add apkg import/export on backend (#1743)
* Add apkg export on backend

* Filter out missing media-paths at write time

* Make TagMatcher::new() infallible

* Gather export data instead of copying directly

* Revert changes to rslib/src/tags/

* Reuse filename_is_safe/check_filename_safe()

* Accept func to produce MediaIter in export_apkg()

* Only store file folder once in MediaIter

* Use temporary tables for gathering

export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.

* Use schedule_as_new() to reset cards

* ExportData → ExchangeData

* Ignore ascii case when filtering system tags

* search_notes_cards_into_table →

search_cards_of_notes_into_table

* Start on apkg importing on backend

* Fix due dates in days for apkg export

* Refactor import-export/package

- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.

* Add SafeMediaEntry for deserialized MediaEntries

* Prepare media based on checksums

- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.

* Handle encoding in `replace_media_refs()`

* Add trait to keep down cow boilerplate

* Add notetypes immediately instaed of preparing

* Move target_col into Context

* Add notes immediately instaed of preparing

* Note id, not guid of conflicting notes

* Add import_decks()

* decks_configs → deck_configs

* Add import_deck_configs()

* Add import_cards(), import_revlog()

* Use dyn instead of generic for media_fn

Otherwise, would have to pass None with type annotation in the default
case.

* Fix signature of import_apkg()

* Fix search_cards_of_notes_into_table()

* Test new functions in text.rs

* Add roundtrip test for apkg (stub)

* Keep source id of imported cards (or skip)

* Keep source ids of imported revlog (or skip)

* Try to keep source ids of imported notes

* Make adding notetype with id undoable

* Wrap apkg import in transaction

* Keep source ids of imported deck configs (or skip)

* Handle card due dates and original due/did

* Fix importing cards/revlog

Card ids are manually uniquified.

* Factor out card importing

* Refactor card and revlog importing

* Factor out card importing

Also handle missing parents .

* Factor out note importing

* Factor out media importing

* Maybe upgrade scheduler of apkg

* Fix parent deck gathering

* Unconditionally import static media

* Fix deck importing edge cases

Test those edge cases, and add some global test helpers.

* Test note importing

* Let import_apkg() take a progress func

* Expand roundtrip apkg test

* Use fat pointer to avoid propogating generics

* Fix progress_fn type

* Expose apkg export/import on backend

* Return note log when importing apkg

* Fix archived collection name on apkg import

* Add CollectionOpWithBackendProgress

* Fix wrong Interrupted Exception being checked

* Add ClosedCollectionOp

* Add note ids to log and strip HTML

* Update progress when checking incoming media too

* Conditionally enable new importing in GUI

* Fix all_checksums() for media import

Entries of deleted files are nulled, not removed.

* Make apkg exporting on backend abortable

* Return number of notes imported from apkg

* Fix exception printing for QueryOp as well

* Add QueryOpWithBackendProgress

Also support backend exporting progress.

* Expose new apkg and colpkg exporting

* Open transaction in insert_data()

Was slowing down exporting by several orders of magnitude.

* Handle zstd-compressed apkg

* Add legacy arg to ExportAnkiPackage

Currently not exposed on the frontend

* Remove unused import in proto file

* Add symlink for typechecking of import_export_pb2

* Avoid kwargs in pb message creation, so typechecking is not lost

Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign 
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.

* Avoid re-exporting protobuf msgs we only use internally

* Stop after one test failure

mypy often fails much faster than pylint

* Avoid an extra allocation when extracting media checksums

* Update progress after prepare_media() finishes

Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.

* Show progress of note imports

Note import is the slowest part, so showing progress here makes the UI
feel more responsive.

* Reset filtered decks at import time

Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.

We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).

https://github.com/ankitects/anki/pull/1743/files#r839346423

* Fix a corner-case where due dates were shifted by a day

This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.

* Log conflicting note in remapped nt case

* take_fields() → into_fields()

* Alias `[u8; 20]` with `Sha1Hash`

* Truncate logged fields

* Rework apkg note import tests

- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)

* Fix sorting of imported decks

Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.

* target[_id]s → existing_card[_id]s

* export_collection_extracting_media() → ...

export_into_collection_file()

* target_already_exists→card_ordinal_already_exists

* Add search_cards_of_notes_into_table.sql

* Imrove type of apkg export selector/limit

* Remove redundant call to mod_schema()

* Parent tooltips to mw

* Fix a crash when truncating note text

String::truncate() is a bit of a footgun, and I've hit this before
too :-)

* Remove ExportLimit in favour of separate classes

* Remove OpWithBackendProgress and ClosedCollectionOp

Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.

Also fix aborting of colpkg exports, which slipped through in #1817.

* Tidy up import log

* Avoid QDialog.exec()

* Default to excluding scheuling for deck list deck

* Use IncrementalProgress in whole import_export code

* Compare checksums when importing colpkgs

* Avoid registering changes if hashes are not needed

* ImportProgress::Collection → ImportProgress::File

* Make downgrading apkgs depend on meta version

* Generalise IncrementableProgress

And use it in entire import_export code instead.

* Fix type complexity lint

* Take count_map for IncrementableProgress::get_inner

* Replace import/export env with Shift click

* Accept all args from update() for backend progress

* Pass fields of ProgressUpdate explicitly

* Move update_interval into IncrementableProgress

* Outsource incrementing into Incrementor

* Mutate ProgressUpdate in progress_update callback

* Switch import/export legacy toggle to profile setting

Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.

One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.

* Show extension in export dialog

* Continue to provide separate options for schema 11+18 colpkg export

* Default to colpkg export when using File>Export

* Improve appearance of combo boxes when switching between apkg/colpkg

+ Deal with long deck names

* Convert newlines to spaces when showing fields from import

Ensures each imported note appears on a separate line

* Don't separate total note count from the other summary lines

This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.

* Fix 'deck not normal' error when importing a filtered deck for the 2nd time

* Fix [Identical] being shown on first import

* Revert "Continue to provide separate options for schema 11+18 colpkg export"

This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.

Will use a different approach

* Move legacy support into a separate exporter option; add to apkg export

* Adjust 'too new' message to also apply to .apkg import case

* Show a better message when attempting to import new apkg into old code

Previously the user could end seeing a message like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

Unfortunately we can't retroactively fix this for older clients.

* Hide legacy support option in older exporting screen

* Reflect change from paths to fnames in type & name

* Make imported decks normal at once

Then skip special casing in update_deck(). Also skip updating
description if new one is empty.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-05-02 21:12:46 +10:00
Damien Elmes
d2ec004ac3 Fix custom study applying previously-selected limit
extend_limits() mutates deck, so we need to re-fetch it.

Closes #1829
2022-05-02 20:57:29 +10:00
Abdo
a3d9a25d18
Extends the SearchNode interface to support field and unqualified searches (#1821)
* Add field search option to SearchNode

* Add unqualified/literal search option to SearchNode

* Rename text to literal_text

* Interpret field name parameter literally
2022-04-28 14:09:51 +10:00
RumovZ
8ccadd96e6
Store last position when scheduling as review (#1800) 2022-04-14 18:53:58 +10:00
Damien Elmes
51a0641bba Sort field change without field change should mark schema modified
Closes #1789
2022-04-10 08:52:55 +10:00
RumovZ
4a6767b1fc
Implicitly group when joining searches (#1759)
* Implicitly group when joining searches

* Allow joining search types directly

* Test search joining

* Add comment for future selves (dae)

* Add one more assert that shows nested grouping (dae)

* Join user searches without grouping again

* Flatten a few clauses in custom study (dae)
2022-04-09 13:22:27 +10:00
Abdo
964f0a5763
Add relative overdueness to review order (#1757)
* Add relative overdueness to review order

* Add test for relative overdue
2022-04-09 13:20:09 +10:00
Damien Elmes
4673fb5fe4 Colpkg import now marks collection as modified
Addresses a corner case where a backup is imported into an
unchanged collection, resulting in the sync indicator showing a full
sync is required, which doesn't go away as syncing thinks no changes
are required.

An alternative way to solve this would be to reverse the order of checks
in the syncing code. It would have the advantage of retaining the
modification time of the backup, but any action like clicking on a deck
would cause it to be updated anyway, so I'm not sure that buys us much.
2022-04-03 16:07:39 +10:00
Damien Elmes
d90608198f Don't rely on frontend to cap time taken in v3 2022-04-02 20:06:23 +10:00
RumovZ
dc36a08f3a
Also restore/keep position of new cards (#1760)
* Also restore/keep position of new cards

* Refactor Card::last_position()
2022-03-31 14:31:13 +10:00
Damien Elmes
9d64afc7bc Handle gaps in media in colpkg imports
Our old Python code was also skipping numbers when it encountered a
directory, leading to a colpkg that couldn't be imported with our new
code.
2022-03-31 13:32:38 +10:00
RumovZ
ab57fa484b
Fix SearchBuilder (#1754)
* Fix missing search grouping

* Fix SearchBuilder::or_join

* Unify search concatenations
2022-03-29 10:53:02 +10:00
RumovZ
f1488b5983
Card type error (#1749)
* TemplateSaveError -> CardTypeError

* Don't show success tooltip if export fails

* Attach help page to error

Show help link if export fails due to card type error.

* Add type (dae)

* Add shared show_exception() (dae)

- Use a shared routine for printing standard backend errors, so that
we can take advantage of the help links in eg. the card layout screen
as well.
- The truthiness check on help in showInfo() would have ignored the
enum 0 value.
- Close the exporting dialog on a documented failure as well

* Fix local variable help_page
2022-03-28 22:17:50 +10:00
RumovZ
dd16890c11
Add Deleted error and disable all bad browser rows (#1742)
* Add Deleted error and disable all bad browser rows

* Avoid error when opening the browse screen to a card with a missing note (dae)

* In cards mode, a missing note is NotFound, not Deleted (dae)

So we distinguish between referential integrity error, and explicit
deletion.

* Remove redundant try block
2022-03-28 19:06:19 +10:00
Damien Elmes
bbcab21f13 Start random card position from 1
Anki's DB schema unfortunately uses odid=0 instead of null to indicate
a lack of an original due date, so having a due position of 0 leads to
the temporary due date not being reset when the card is removed from
a filtered deck.

https://forums.ankiweb.net/t/anki-2-1-50-beta-6-9-stable-release/18181/52
2022-03-23 22:00:25 +10:00
Damien Elmes
6735b23e5b Force a schema change on colpkg import
If we don't force a full sync when restoring, any items that were added
since the backup may have already been sent to AnkiWeb, and they
won't have deletion records. After the user restores from a backup,
they'll end up in a state where their local and AnkiWeb collections
differ, and the changes will not sync. The old backup code forced a schema
change, but we weren't previously doing it via File>Import.
2022-03-22 13:10:32 +10:00
Damien Elmes
4515c41d2c
Backup improvements (#1728)
* Collection needs to be closed prior to backup even when not downgrading

* Backups -> BackupLimits

* Some improvements to backup_task

- backup_inner now returns the error instead of logging it, so that
the frontend can discover the issue when they await a backup (or create
another one)
- start_backup() was acquiring backup_task twice, and if another thread
started a backup between the two locks, the task could have been accidentally
overwritten without awaiting it

* Backups no longer require a collection close

- Instead of closing the collection, we ensure there is no active
transaction, and flush the WAL to disk. This means the undo history
is no longer lost on backup, which will be particularly useful if we
add a periodic backup in the future.
- Because a close is no longer required, backups are now achieved with
a separate command, instead of being included in CloseCollection().
- Full sync no longer requires an extra close+reopen step, and we now
wait for the backup to complete before proceeding.
- Create a backup before 'check db'

* Add File>Create Backup

https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157

* Defer checkpoint until we know we need it

When running periodic backups on a timer, we don't want to be fsync()ing
unnecessarily.

* Skip backup if modification time has not changed

We don't want the user leaving Anki open overnight, and coming back
to lots of identical backups.

* Periodic backups

Creates an automatic backup every 30 minutes if the collection has been
modified.

If there's a legacy checkpoint active, tries again 5 minutes later.

* Switch to a user-configurable backup duration

CreateBackup() now uses a simple force argument to determine whether
the user's limits should be respected or not, and only potentially
destructive ops (full download, check DB) override the user's configured
limit.

I considered having a separate limit for collection close and automatic
backups (eg keeping the previous 5 minute limit for collection close),
but that had two downsides:

- When the user closes their collection at the end of the day, they'd
get a recent backup. When they open the collection the next day, it
would get backed up again within 5 minutes, even though not much had
changed.
- Multiple limits are harder to communicate to users in the UI

Some remaining decisions I wasn't 100% sure about:

- If force is true but the collection has not been modified, the backup
will be skipped. If the user manually deleted their backups without
closing Anki, they wouldn't get a new one if the mtime hadn't changed.
- Force takes preference over the configured backup interval - should
we be ignored the user here, or take no backups at all?

Did a sneaky edit of the existing ftl string, as it hasn't been live
long.

* Move maybe_backup() into Collection

* Use a single method for manual and periodic backups

When manually creating a backup via the File menu, we no longer make
the user wait until the backup completes. As we continue waiting for
the backup in the background, if any errors occur, the user will get
notified about it fairly quickly.

* Show message to user if backup was skipped due to no changes

+ Don't incorrectly assert a backup will be created on force

* Add "automatic" to description

* Ensure we backup prior to importing colpkg if collection open

The backup doesn't happen when invoked from 'open backup' in the profile
screen, which matches Anki's previous behaviour. The user could
potentially clobber up to 30 minutes of their work if they exited to
the profile screen and restored a backup, but the alternative is we
create backups every time a backup is restored, which may happen a number
of times if the user is trying various ones. Or we could go back to a
separate throttle amount for this case, at the cost of more complexity.

* Remove the 0 special case on backup interval; minimum of 5 minutes

https://github.com/ankitects/anki/pull/1728#discussion_r830876833
2022-03-21 19:40:42 +10:00
Damien Elmes
e2eff7deae Fix change notetype affecting cards outside selection
Accidentally introduced by the search refactoring in #1600, this lead
to a query that matched items outside of the selected notes, eg

(n.id in (1506029488152) and c.ord = 43 or c.ord = 4)

Closes #1727
2022-03-20 15:36:54 +10:00
Damien Elmes
d100f7a2c8 Don't fsync media files on import
I was seeing import speeds of only 10-20 files a second before this
change.
2022-03-19 23:31:11 +10:00
RumovZ
16fe18d033
Refactor export-import code and resolve fixmes (#1723)
* Write media files in chunks

* Test media file writing

* Add iter `ReadDirFiles`

* Remove ImportMediaError, fail fatally instead

Partially reverts commit f8ed4d89ba.

* Compare hashes of media files to be restored

* Improve `MediaCopier::copy()`

* Restore media files atomically with tempfile

* Make downgrade flag an enum

* Remove SchemaVersion::Latest in favour of Option

* Remove sha1 comparison again

* Remove unnecessary repr(u8) (dae)
2022-03-18 19:31:55 +10:00
RumovZ
5781e86995
Don't propogate limit extensions in v3 scheduler (#1724) 2022-03-18 19:16:31 +10:00
Damien Elmes
5c1abb7f09 Add missing file header 2022-03-17 21:09:20 +10:00
Damien Elmes
dadb69a14b Fix some clippy lints + imports 2022-03-17 21:03:39 +10:00
Damien Elmes
bf8e70c70f Ensure partial colpkg file removed if export fails 2022-03-17 20:58:36 +10:00
Damien Elmes
f8ed4d89ba Add new error variant to frontend; ensure errors are mapped appropriately 2022-03-17 20:58:35 +10:00
Damien Elmes
30e53c51ab Tiny cleanup missing from previous commit
Had intended to apply it prior to pushing.
2022-03-17 17:49:18 +10:00
Damien Elmes
4687620f5e Expand normalization checks on import/export
The old Python code was only checking for NFC encoding, but we should
check for other issues like special filenames on windows (eg con.mp3)

- On export, the user is told to use Check Media if their media has
invalid filenames.
- On import, legacy packages will be transparently normalized. Since we're
doing the checks on export as well, any invalid names in a v3 package
are an error.
2022-03-17 17:31:19 +10:00
Damien Elmes
57a4495d92 Check for attempted path traversal on import 2022-03-17 16:40:52 +10:00
Damien Elmes
c2e8d89fc6
Colpkg fixes (#1722)
* Fix legacy colpkg import; disable v3 import/export; add roundtrip test

The test has revealed we weren't decompressing the media files on v3
import. That's easy to fix, but means all files need decompressing
even when they already exist, which is not ideal - it would be better
to store size/checksum in the metadata instead.

* Switch media and meta to protobuf; re-enable v3 import/export

- Fixed media not being decompressed on import
- The uncompressed size and checksum is now included for each media
entry, so that we can quickly check if a given file needs to be extracted.
We're still just doing a naive size comparison on colpkg import at the
moment, but we may want to use a checksum in the future, and will need
a checksum for apkg imports.
- Checksums can't be efficiently encoded in JSON, so the media list
has been switched to protobuf to reduce the the space requirements.
- The meta file has been switched to protobuf as well, for consistency.
This will mean any colpkg files exported with beta7 will be
unreadable.

* Avoid integer version comparisons

* Re-enable v3 test

* Apply suggestions from code review

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>

* Add export_colpkg() method to Collection

More discoverable, and easier to call from unit tests

* Split import/export code out into separate folders

Currently colpkg/*.rs contain some routines that will be useful for
apkg import/export as well; in the future we can refactor them into a
separate file in the parent module.

* Return a proper error when media import fails

This tripped me up when writing the earlier unit test - I had called
the equivalent of import_colpkg()?, and it was returning a string error
that I didn't notice. In practice this should result in the same text
being shown in the UI, but just skips the tooltip.

* Automatically create media folder on import

* Move roundtrip test into separate file; check collection too

* Remove zstd version suffix

Prevents a warning shown each time Rust Analyzer is used to check the
code.

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
2022-03-17 15:11:23 +10:00
Damien Elmes
ed9476c856 Update Rust deps
Primarily for https://blog.rust-lang.org/2022/03/08/cve-2022-24713.html
2022-03-15 16:51:52 +10:00
RumovZ
e759885734
Backend colpkg exporting (#1719)
* Implement colpkg exporting on backend

* Use exporting logic in backup.rs

* Refactor exporting.rs

* Add backend function to export collection

* Refactor backend/collection.rs

* Use backend for colpkg exporting

* Don't use default zip compression for media

* Add exporting progress

* Refactor media file writing

* Write dummy collections

* Localize dummy collection note

* Minimize dummy db size

* Use `NamedTempFile::new()` instead of `new_in`

* Drop redundant v2 dummy collection

* COLLECTION_VERSION -> PACKAGE_VERSION

* Split `lock_collection()` into two to drop flag

* Expose new colpkg in GUI

* Improve dummy collection message

* Please type checker

* importing-colpkg-too-new -> exporting-...

* Compress the media map in the v3 package (dae)

On collections with lots of media, it can grow into megabytes.

Also return an error in extract_media_file_names(), instead of masking
it as an optional.

* Store media map as a vector in the v3 package (dae)

This compresses better (eg 280kb original, 100kb hashmap, 42kb vec)

In the colpkg import case we don't need random access. When importing
an apkg, we will need to be able to fetch file data for a given media
filename, but the existing map doesn't help us there, as we need
filename->index, not index->filename.

* Ensure folders in the media dir don't break the file mapping (dae)
2022-03-15 16:48:02 +10:00
Damien Elmes
2d6dd0630f Switch card.original_position to a proto3 optional 2022-03-10 20:48:26 +10:00
Damien Elmes
f3e81c8a95 Move custom study tag and limit gathering+saving into the backend
Ideally this would have been in beta 6 :-) No add-ons appear to be
using customstudy.py/taglimit.py though, so it should hopefully not be
disruptive.

In the earlier custom study changes, we didn't get around to addressing
issue #1136. Now instead of trying to determine the maximum increase
to allow (which doesn't work correctly with nested decks), we just
present the total available to the user again, and let them decide. There's
plenty of room for improvement here still, but further work here might
be better done once we look into decoupling deck limits from deck presets.

Tags and available cards are fetched prior to showing the dialog now,
and will show a progress dialog if things take a while.

Tags are stored in an aux var now, so they don't inflate the deck
object size.
2022-03-10 16:23:03 +10:00
RumovZ
b9c3b12f71
Optionally restore original position and reset counts when forgetting (#1714)
* Add forget prompt with options

- Restore original position
- Reset reps and lapses

* Restore position when resetting for export

* Add config context to avoid passing keys

* Add routine to fetch defaults; use method-specific enum (dae)

* Keep original position by default (dae)

* Fix code completion for forget dialog (dae)

Needs to be a symbolic link to the generated file
2022-03-09 16:51:41 +10:00
RumovZ
c21e6e2b97
Disable full screen on Windows with OpenGL (#1715) 2022-03-09 14:21:54 +10:00
Damien Elmes
e6028cd9a4 Preserve old translation
7f076ee92a..9f37a0245c (r820447434)
2022-03-07 18:14:12 +10:00
RumovZ
f3c8857421
Backups (#1685)
* Add zstd dep

* Implement backend backup with zstd

* Implement backup thinning

* Write backup meta

* Use new file ending anki21b

* Asynchronously backup on collection close in Rust

* Revert "Add zstd dep"

This reverts commit 3fcb2141d2be15f907269d13275c41971431385c.

* Add zstd again

* Take backup col path from col struct

* Fix formatting

* Implement backup restoring on backend

* Normalize restored media file names

* Refactor `extract_legacy_data()`

A bit cumbersome due to borrowing rules.

* Refactor

* Make thinning calendar-based and gradual

* Consider last kept backups of previous stages

* Import full apkgs and colpkgs with backend

* Expose new backup settings

* Test `BackupThinner` and make it deterministic

* Mark backup_path when closing optional

* Delete leaky timer

* Add progress updates for restoring media

* Write restored collection to tempfile first

* Do collection compression in the background thread

This has us currently storing an uncompressed and compressed copy of
the collection in memory (not ideal), but means the collection can be
closed without waiting for compression to complete. On a large collection,
this takes a close and reopen from about 0.55s to about 0.07s. The old
backup code for comparison: about 0.35s for compression off, about
8.5s for zip compression.

* Use multithreading in zstd compression

On my system, this reduces the compression time of a large collection
from about 0.55s to 0.08s.

* Stream compressed collection data into zip file

* Tweak backup explanation

+ Fix incorrect tab order for ignore accents option

* Decouple restoring backup and full import

In the first case, no profile is opened, unless the new collection
succeeds to load.
In the second case, either the old collection is reloaded or the new one
is loaded.

* Fix number gap in Progress message

* Don't revert backup when media fails but report it

* Tweak error flow

* Remove native BackupLimits enum

* Fix type annotation

* Add thinning test for whole year

* Satisfy linter

* Await async backup to finish

* Move restart disclaimer out of backup tab

Should be visible regardless of the current tab.

* Write restored collection in chunks

* Refactor

* Write media in chunks and refactor

* Log error if removing file fails

* join_backup_task -> await_backup_completion

* Refactor backup.rs

* Refactor backup meta and collection extraction

* Fix wrong error being returned

* Call sync_all() on new collection

* Add ImportError

* Store logger in Backend, instead of creating one on demand

init_backend() accepts a Logger rather than a log file, to allow other
callers to customize the logger if they wish.

In the future we may want to explore using the tracing crate as an
alternative; it's a bit more ergonomic, as a logger doesn't need to be
passed around, and it plays more nicely with async code.

* Sync file contents prior to rename; sync folder after rename.

* Limit backup creation to once per 30 min

* Use zstd::stream::copy_decode

* Make importing abortable

* Don't revert if backup media is aborted

* Set throttle implicitly

* Change force flag to minimum_backup_interval

* Don't attempt to open folders on Windows

* Join last backup thread before starting new one

Also refactor.

* Disable auto sync and backup when restoring again

* Force backup on full download

* Include the reason why a media file import failed, and the file path

- Introduce a FileIoError that contains a string representation of
the underlying I/O error, and an associated path. There are a few
places in the code where we're currently manually including the filename
in a custom error message, and this is a step towards a more consistent
approach (but we may be better served with a more general approach in
the future similar to Anyhow's .context())
- Move the error message into importing.ftl, as it's a bit neater
when error messages live in the same file as the rest of the messages
associated with some functionality.

* Fix importing of media files

* Minor wording tweaks

* Save an allocation

I18n strings with replacements are already strings, so we can skip the
extra allocation. Not that it matters here at all.

* Terminate import if file missing from archive

If a third-party tool is creating invalid archives, the user should know
about it. This should be rare, so I did not attempt to make it
translatable.

* Skip multithreaded compression on small collections

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-03-07 15:11:31 +10:00
Damien Elmes
864904729b Fix an intermittent failure in one of the Rust unit tests 2022-03-07 14:18:17 +10:00
RumovZ
5426164ebf
Add regex tag search (#1707) 2022-03-04 18:43:27 +10:00
Damien Elmes
2c80bcb571 Update to latest rules_rust, which unblocks the tokio upgrade
Rust version has been pinned, as 1.59 gets stuck compiling pyo3_macros,
and appears to have other issues:
https://github.com/rust-lang/rust/issues/94390
2022-03-03 19:48:17 +10:00
RumovZ
508b5ab947
Remove top_deck_id arg in deck_tree() (#1702)
Counts don't propogate correctly anymore (#1678).
2022-03-02 15:30:32 +10:00
Damien Elmes
6941bccde4 Add support for proto3 optional scalars
Protobuf 3.15 introduced support for marking scalar fields like
uint32 as optional, and all of our tooling appears to support it
now. This allows us to use simple optional/null checks in our Rust/
TypeScript code, without having to resort to an inner message.

I had to apply a minor patch to protobufjs to get this working with
the json-module output; this has also been submitted upstream:
https://github.com/protobufjs/protobuf.js/pull/1693

I've modified CardStatsResponse as an example of the new syntax.

One thing to note: while the Rust and TypeScript bindings use optional/
null fields, as that is the norm in those languages, Google's Python
bindings are not very Pythonic. Referencing an optional field that is
missing will yield the default value, and a separate HasField() call
is required, eg:

```
>>> from anki.stats_pb2 import CardStatsResponse as R
... msg = R.FromString(b"")
... print(msg.first_review)
... print(msg.HasField("first_review"))
0
False
```
2022-02-27 19:42:06 +10:00
Damien Elmes
89bce6d829 Update Rust deps; pin Tokio
Tokio has had to be pinned, because the 1.17 release introduces
a dependency on windows_sys, which fails to build on Windows on
Bazel.

The issue appears to be the build script of a subcrate - it is using
CARGO_MANIFEST_DIR to update the linking path so windows.lib can be
found (it's contained in that crate), but the path is set incorrectly.

dfc25285a2/crates/targets/x86_64_msvc/build.rs

One way we might be able to work around it is to add to the link path
in our own build script.
2022-02-25 17:10:47 +10:00
Damien Elmes
d52c36e920 Make flag changes undoable again
The previous change in 1871b57663 failed
to consider the browser refreshing case, as reported here:
https://forums.ankiweb.net/t/anki-2-1-50-beta-3-4/17501/30

I previously attempted to solve this by having SetFlag skip the queue
rebuild, then mutating the captured mtimes in the queues. That didn't
work correctly when undoing, as the queue mutations weren't recorded.
This approach combines that attempt and the previous change: flag
setting is an undoable operation again, but does not change the card's
modification time, so it can be applied/undone without a queue build
being required. Instead of special-casing flag changes in the review
screen, we now just redraw the flag on changes.card, as any other card
op will have triggered a queue rebuild.
2022-02-25 15:06:07 +10:00
RumovZ
a4d61fe7d9
Original position (#1677)
* Replace Card.data with .original_position

* Use and update original position in v3

* Show original position in card info

* Revert restoring original position for now

* Fix pb card to/from pylib card

* Try original_position as the last pb field

* minor wording tweaks (dae)
2022-02-22 22:48:21 +10:00
RumovZ
131a94dd85
Config for burying interday learning cards (#1680)
* Add config for burying interday learning cards

* Expose bury interday learning config in GUI
2022-02-22 21:37:59 +10:00
Abdo
fd6a78e7cf
Add option to ignore accents in search by default (#1667)
* Add option to ignore accents in unqualified search

* Support ignoring accents in word boundary search

* Update ftl/core/preferences.ftl
2022-02-17 16:30:52 +10:00
Damien Elmes
1871b57663 don't put flag changes in the undo queue, and don't bump mtime
This is not ideal, but I struggled to come up with a better solution.

Background:

- The scheduler records the mtime of cards as it's building the queues,
and will throw an error in get_queued_cards() if the card on the DB
has a different mtime. This is to catch bugs - any operation that modifies
cards should be triggering a queue rebuild, or should adjust the queues
appropriately.
- The review screen skips the usual queue rebuild redraw, and directly
updates the flag icon. This is because a rebuild could cause a different
card to appear, or the answer side to switch back to the question side,
neither of which the user expects when they flag a card.

The current behaviour was broken: the queue rebuilding was still happening
on the backend, and the frontend was just failing to reflect it.

I initially tried to special-case Op::SetFlag, having it skip the queue
rebuild, and having set_card_flag() update the mtimes in the active
queue. But those mutations weren't captured by the undo log, so they
didn't get undone when undoing the set flag operation. We could perhaps
work around it by adding a separate undo entry to capture the mutation,
but it started to feel like it would be a pain to maintain moving forward.

By skipping the undo queue and retaining the same mtime, no queue
rebuild is required. Because we're setting usn, the cards will still
sync, but as mtime is not bumped, in the case of a conflict, an older
unsynced change from another client may revert the flag change.

Fixes https://forums.ankiweb.net/t/anki-2-1-50-beta-1-2/15608/145
2022-02-13 16:27:33 +10:00
RumovZ
6a21261c84
Round calculated Hard days (#1661)
* Round calculated Hard days

* clarify docstring (dae)
2022-02-13 13:37:52 +10:00
RumovZ
d55f080733
V3 parent limits (#1638)
* avoid repinning Rust deps by default

* add id_tree dependency

* Respect intermediate child limits in v3

* Test new behaviour of v3 counts

* Rework v3 queue building to respect parent limits

* Add missing did field to SQL query

* Fix `LimitTreeMap::is_exhausted()`

* Rework tree building logic

https://github.com/ankitects/anki/pull/1638#discussion_r798328734

* Add timer for build_queues()

* `is_exhausted()` -> `limit_reached()`

* Move context and limits into `QueueBuilder`

This allows for moving more logic into QueueBuilder, so less passing
around of arguments. Unfortunately, some tests will require additional
work to set up.

* Fix stop condition in new_cards_by_position

* Fix order gather order of new cards by deck

* Add scheduler/queue/builder/burying.rs

* Fix bad tree due to unsorted child decks

* Fix comment

* Fix `cap_new_to_review_rec()`

* Add test for new card gathering

* Always sort `child_decks()`

* Fix deck removal in `cap_new_to_review_rec()`

* Fix sibling ordering in new card gathering

* Remove limits for deck total count with children

* Add random gather order

* Remove bad sibling order handling

All routines ensure ascending order now.
Also do some other minor refactoring.

* Remove queue truncating

All routines stop now as soon as the root limit is reached.

* Move deck fetching into `QueueBuilder::new()`

* Rework new card gather and sort options

https://github.com/ankitects/anki/pull/1638#issuecomment-1032173013

* Disable new sort order choices ...

depending on set gather order.

* Use enum instead of numbers

* Ensure valid sort order setting

* Update new gather and sort order tooltips

* Warn about random insertion order with v3

* Revert "Add timer for build_queues()"

This reverts commit c9f5fc6ebe87953c17a0c842990b009b5596c69c.

* Update rslib/src/storage/card/mod.rs (dae)

* minor wording tweaks to the tooltips (dae)

+ move legacy strings to bottom
+ consistent capitalization (our leech action still needs fixing,
but that will require introducing a new 'suspend card' string as the
existing one is used elsewhere as well)
2022-02-10 09:55:43 +10:00
RumovZ
4909302147
Intersperse more evenly, mainly at start and end (#1651)
Closes #1645.
2022-02-09 12:45:37 +10:00
RumovZ
872b6df22a
Optimise searching in (all) fields (#1622)
* Avoid rebuilding regex in field search

* Special case search in all fields

* Don't repeat mid nodes in field search sql

Small speed gain for searches like `*:re:foo` and reduces the sql tree
depth if a lot of field names of the same notetype match.

* Add sql function to match fields with regex

* Optimise used field search algorithm

- Searching in all fields is a special case.
- Using native SQL comparison is preferred.
- For Regex, use newly added SQL function.

* Please clippy

* Avoid pyramid of doom

* nt_fields -> matched_fields

* Add tests for regex and all field searches

* minor tweaks for readability (dae)
2022-01-24 20:30:08 +10:00
RumovZ
d4e3cc7a0a
Remove into_node_list() (#1609) 2022-01-21 21:23:52 +10:00
RumovZ
9aca778a93
Backend Custom Study (#1600)
* Implement custom study on backend

* Switch frontend to backend custom study

* Skip typecheck for new pb classes

* Build tag search string on backend

Also fixes escaping of special characters in tag names.

* `cram.cards` -> `cram.card_limit`

* Assign more meaningful names in `TagLimit`

* Broaden rustfmt glob

* Use `invalid_input()` helper

* Assign `FilteredDeckForUpdate` to temp var

* Implement `SearchBuilder`

* Rewrite `custom_study()` with `SearchBuilder`

* Replace match macros with `SearchBuilder`

* Remove `into_nodes_list` & `concatenate_searches`
2022-01-20 14:25:22 +10:00
RumovZ
1dae2f42ff
Make note: and card: search require full match (#1606) 2022-01-20 11:57:16 +10:00
RumovZ
77b1546583
Remove redundant error mapping (#1593)
Helpful regex error in Find&Replace was obscured by generic
"Invalid input" error.
2022-01-16 13:46:27 +10:00
Damien Elmes
bbc191711d update Rust deps 2022-01-15 14:59:43 +10:00
Abdo
aaaea03c72
Fix new preview card's position being interpreted as a date (#1577)
* Fix new preview card's position being interpreted as a date

Can be reproduced by opening the Card Info screen of a new preview card
not answered yet.

* Update rslib/src/stats/card.rs
2022-01-06 19:52:54 +10:00
Damien Elmes
693672d7a3 remove extraneous to_vec() call 2021-12-24 13:09:19 +10:00
Damien Elmes
b7907741ec fix SQL formatting issue 2021-12-24 12:42:42 +10:00
Damien Elmes
8ed1fa7e99 fix interday learning cards not being buried
https://forums.ankiweb.net/t/2-1-49-mac-automatic-unbury-still-not-working/14799/43
2021-12-24 12:25:16 +10:00
Damien Elmes
4d431fb7af move linkchecker into separate crate
The feature-based approach didn't work with cargo-raze, leading
to ./update.py in cargo/ breaking.
2021-12-20 17:27:43 +10:00
RumovZ
3e0c9dc866
New TTS/AV tag handling (#1559)
* Add new `card_rendering` mod

Parses a text with av/tts tags and strips or extracts tags.

* Replace old `extract_av_tags` and `strip_av_tags`

... with new `card_rendering` mod

* ressource -> resource

* Add AV prettifier for use in browser table

* Accept String in av tag routines

... and avoid redundant writes if no changes need to be made.

* add benchmarking with criterion; make links test optional (dae)

cargo install cargo-criterion, then run ./bench.sh

* performance comparison: creating HashMap up front (dae)

the previous solution:

anki_tag_parse          time:   [1.8401 us 1.8437 us 1.8476 us]

this solution:

anki_tag_parse          time:   [2.2420 us 2.2447 us 2.2477 us]
                        change: [+21.477% +21.770% +22.066%] (p = 0.00 < 0.05)
                        Performance has regressed.

* Revert "performance comparison: creating HashMap up front" (dae)

This reverts commit f19126a2f15b729b825825a49283f63ab13474d0.

* add missing header

* Write error message if tts lang is missing

* `Tag` -> `Directive`
2021-12-17 19:04:42 +10:00
RumovZ
2221d0a520
Make Hard use current step's interval if it's not the first one (#1561)
* Make hard repeat the current step's interval in v3

Unless for the first step to avoid identical interval with Again.

* Make Hard repeat the current step's interval in v2

* Adjust test to new Hard behaviour
2021-12-16 22:02:13 +10:00
RumovZ
80ed94ed08
Cap steps (#1548)
* Fix steps being mistaken for seconds

* Cap steps at `u32::max` seconds

* Fix overflow of steps in Rust

* Prevent overflow of `IntervalKind`

* Prevent overflow in `revlod/mod.rs`

Also replace some `as` with `from` and `try_from` as is recommended to
highlight potential issues.

* Ensure v2 doesn't store overflowing revlog ivls

* Lower steps cap in deck options

Whereas large card intervals are converted to days, revlog intervals use
i32s to store large numbers of seconds.

* Format
2021-12-15 18:46:26 +10:00
Damien Elmes
8d90b6b061 run buildifier/buildozer to tidy up BUILD files 2021-12-14 09:18:24 +10:00
Damien Elmes
0c821859c5 check for missed dbg!() statements
This works in our main code, but doesn't catch dbg!() statements in
our build scripts.
2021-12-14 09:05:14 +10:00
Damien Elmes
afff4fc437 order by template after note id in filtered decks
This brings the behaviour a bit closer to the default ordering of new
cards when they are reset, and is better than an undefined template
order. But it's a stopgap solution, and in the long run, filtered decks
need a bit of a rethink with the improved ordering than v3 has brought.
2021-12-10 18:53:50 +10:00
Damien Elmes
d515939340 fix error when gathering new cards in reverse position
Also simplify order clause - with did and queue limited to a constant,
SQLite is smart enough to use the covering index for sorting by due.
2021-12-06 17:08:00 +10:00
Damien Elmes
c2ab0de34f update Rust deps 2021-12-03 20:35:53 +10:00
Damien Elmes
67ee6f9c0e update to Rust 1.57 + latest rules_rust 2021-12-03 20:35:52 +10:00
Damien Elmes
8de3eaea65 fix Clippy lints in Rust 1.57 2021-12-03 19:53:37 +10:00
Damien Elmes
d7f5555642 deck_tree() could be made to unbury the current day's cards
When provided with a future timestamp to estimate the next day's cards,
it lead to the current day's buried cards being unburied.

Fixes
https://forums.ankiweb.net/t/inconsistent-card-counts-when-syncing/15496
2021-12-02 21:07:21 +10:00
Damien Elmes
9daa0c9acb fix new cards not appearing in correct order in v3
This was broken by an SQLite upgrade - previously we received the rows
in ix_cards_sched order, but recent versions use a table scan for that
query when the order is unspecified. Solved by being explicit about the
order we expect results to arrive.

https://forums.ankiweb.net/t/skipping-new-cards/15410
2021-12-02 17:20:48 +10:00
Matthias Metelka
68092082f2
Change Notetype UI Rework (#1499)
* Enable access to old notetype name

* Set minimum height for ChangeNotetypeDialog

* Add bootstrap icons to change-notetype

* Move alert up and make it collapsible

* Tweak some CSS

- Add variables --sticky-bg and --sticky-border to StickyContainer
- Tweak base.css

* Add translatable string "(Nothing)"

* Rework ChangeNotetype screen

* Initially load option at newIndex and remaining options on focus

Optimization for big notetypes:
Should increase efficiency from O(n²) to O(n). Test on notetype with 500 templates shows significant improvement in load time (~10s down to ~1s).

* Try to satisfy rust test

* Change arrow direction depending on reading direction

+ add 0.5em top padding to main

* Create Alert.svelte

* Introduce CSS variable --pane-bg

* Revert "Initially load option at newIndex and remaining options on focus"

This reverts commit f42beee45c27dba9433d76217fb583b117fb5231.

* Final cleanup

* Refine padding/gutter
2021-11-24 12:09:55 +10:00
Henrik Giesel
2778b9220c
Mathjax editor improvements (#1502)
* Remove unnecessary stopPropagation of mathjax-overlay events

* Use CodeMirror component for MathjaxHandle

* Refactor ResizeObserver code in MathjaxHandle

* Wrap setRange in CodeMirror in try/catch

* Add Mathjax Editor bottom margin

* Add custom Enter and Shift+Enter shortcuts for the MathjaxHandle

* Format

* Move placeCaretAfter to domlib

* Move focus back to field after editing Mathjax

* Put Cursor after Mathjax after accepting

* Add delete button for Mathjax

* Change border color of mathjax menu

* Refactor into MathjaxMenu

* Put caretKeyword in variable

* Use one ResizeObserver for all Mathjax images

* Add minmimum width for Mathjax editor

* is still smaller than minimal window width

* Add bazel directories to .prettierignore and format from root

* exclude ftl/usage (dae)

the json files that live there are output from our tooling, and
formatting them means an extra step each time we want to update them

also exclude .mypy_cache, which is output by scripts/mypy*

* minor ftl tweak: newline -> new line  (dae)
2021-11-23 10:27:32 +10:00
Damien Elmes
11042a16ce Fix latest Rusqlite on Windows
https://github.com/bazelbuild/rules_rust/pull/1025
2021-11-19 10:44:02 +10:00
Damien Elmes
8b3d964866 roll rusqlite back due to Windows breakage 2021-11-18 21:39:23 +10:00
Damien Elmes
d31dccc388 update Rust deps 2021-11-18 20:58:41 +10:00
Damien Elmes
26123ce229 update to edition 2021 2021-11-18 20:51:10 +10:00
RumovZ
72b08cb7d5
Smooth fuzz (#1493)
* Add separate `fuzz.rs`

* Switch to a smoother fuzz calculation
2021-11-17 07:23:19 +10:00
Damien Elmes
13382f7f87 avoid fuzzing until interval reaches 3 days
e5e47a31fe (r748827327)

+ switched assert_lower_middle_upper to a macro, so that when it fails,
the reported line number is the original call site, instead of one inside
the helper function
2021-11-15 15:48:58 +10:00
RumovZ
b1142c12d9
Fix constrained_fuzz_bounds() (#1490)
... for cases where the entire fuzz range is above `maximum`.
Also improve hanling if the entire range is below `minimum` and
readability.
2021-11-15 15:41:43 +10:00
Damien Elmes
311e70822d fix a link anchor 2021-11-14 11:44:54 +10:00
Damien Elmes
5f05eeb922 unbury when refreshing queues
While we already unbury when refreshing the deck list, if the user
resumes study on a new day without refreshing the deck list, burying
could end up being delayed.

Possible fix for https://forums.ankiweb.net/t/buried-cards-in-ankimobile-beta-20081-3/14753/3
2021-11-14 10:06:47 +10:00
Damien Elmes
044d253306 fix underflow in fuzz code, leading to large intervals
https://forums.ankiweb.net/t/buried-cards-in-ankimobile-beta-20081-3/14753
2021-11-14 09:17:37 +10:00
RumovZ
0efa3f944f
Garbage collect unused Fluent strings (#1482)
* Canonify import of i18n module

Should always be imported as `tr`, or `tr2` if there is a name collision
(Svelte).

* Add helper for garbage collecting ftl strings

Also add a serializer for ftl asts.

* Add helper for filter-mapping `DirEntry`s

* Fix `i18n_helpers/BUILD.bazel`

* run cargo-raze

* Refactor `garbage_collection.rs`

- Improve helper for file iterating
- Remove unused terms as well
- Fix issue with checking for nested messages by switching to a regex-
based approach (which runs before deleting)
- Some more refactorings and lint fixes

* Fix lints in `serialize.rs`

* Write json pretty and sorted

* Update `serialize.rs` and fix header

* Fix doc and remove `dbg!`

* Add binaries for ftl garbage collection

Also relax type constraints and strip debug tests.

* add rust_binary targets for i18n helpers (dae)

* add scripts to update desktop usage/garbage collect (dae)

Since we've already diverged from 2.1.49, we won't gain anything
from generating a stable json just yet. But once 2.1.50 is released,
we should run 'ftl/update-desktop-usage.sh stable'.

* add keys from AnkiMobile (dae)

* Mention caveats in `remove-unused.sh`
2021-11-12 18:19:01 +10:00
Damien Elmes
cb07b232d8 add a builder for Collection 2021-11-06 14:43:41 +10:00
RumovZ
283776d8e7
Rework v3 fuzzing (#1474)
* Remove flooring in v3 scheduler code

It is no longer supposed to be an exact port of the old Python code.

* Rework v3 fuzzing

https://github.com/ankitects/anki/issues/1416#issuecomment-958208149

* Ensure length of fuzz range is larger than 1

Only for new intervals larger than 1 and respecting max review interval.

* add the beginnings of a unit test

* Clarify `fuzz_factor` doc string

* Fix Python tests for 2021 scheduler

* Fix fuzz test

1.0 is not a valid fuzz factor.

* Add tests for fuzzing in Rust

* Use range notation in fuzz factor doc

* Strip redundant tests
2021-11-06 10:39:24 +10:00
Matthias Metelka
371f731e30
Editor Field Descriptions (#1476)
* Add description input to fields dialog

QLineEdit seems like the best option, as it saves space and motivates users to keep their descriptions concise.

* Add setDescriptions to note initialization script

Went for the extra function instead of including it in setFields to prevent potential add-on breakages.

* Add tooltip next to field name if description is set

* Refactor code according to suggestions

Set default tooltip placement to right instead of bottom

Use .get() for fld["description"]

Fix tab order in fields dialog

Swap out abbreviation "desc" for full length name to keep consistency

* Update Protobuf and Rust for description

Add description to notetypes.proto and schema11

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>

* Fix tooltips not updating with description

Remove redundant variable tooltipOptions

Update previousTooltip within reactive function

* Move LabelDescription out of LabelName

Co-authored-by: Henrik Giesel <hgiesel@users.noreply.github.com>

* Decrease icon size and fix alignment

Co-Authored-By: Henrik Giesel <hengiesel@gmail.com>

* the new key needs to be cleared from fields, not the notetype itself

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>
Co-authored-by: Henrik Giesel <hengiesel@gmail.com>
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2021-11-06 09:42:48 +10:00
Damien Elmes
3843a4509e fix sort order toggling
We're getting an enum instead of an int in Qt6

normal/reversed have been renamed to ascending/descending; no add-ons
appear to be using the old versions.
2021-10-30 09:14:43 +10:00
RumovZ
16ad0137f7
Use backend for getting deck children (#1465)
* Add backend routine for child deck names and ids

* Use backend for getting deck children

Instead of flawed case-sensitive Python logic.
2021-10-29 19:43:17 +10:00
Damien Elmes
0e00c4a461 fix new cards not being correctly limited
https://forums.ankiweb.net/t/ios-beta-20080-2-more-new-cards-after-review-limit-is-met/13728/10
2021-10-29 12:12:34 +10:00
RumovZ
7e23c9fe4f
Fix quotation of "and" and "or" in search (#1463) 2021-10-28 19:23:56 +10:00
RumovZ
3ab9712c18
Stop trimming filename references before encoding (#1462)
Closes  #1430
2021-10-28 19:22:51 +10:00
Damien Elmes
e30411e0b2 update Rust deps with cargo-raze 0.13.0 2021-10-26 08:16:40 +10:00
RumovZ
3cdb3d72c1
Do not bury suspended cards (#1447)
* Skip burying for suspended cards

* Inform about number of buried cards
2021-10-23 11:04:26 +10:00
RumovZ
1c9b5a2e83
Card info cleanup (#1446)
* Cast proto interface to type ...

... instead of using non-null assertions in Revlog.svelte.

* Remove OptionalInt32 and OptionalUInt32
2021-10-23 11:00:43 +10:00
RumovZ
56d8402f89
Fix underflow of learning count (#1444)
`counts.learning` includes interday learning cards, so it is not
suitable to determine how many cards from the (intraday!) learning queue
are already included in the learning count when updating it.
2021-10-22 20:58:06 +10:00
Damien Elmes
61f3b71664 fix a clippy lint in 1.56 2021-10-22 12:03:54 +10:00
Damien Elmes
3d416b4a68 convert sql_format to ts_project; move into separate folder 2021-10-18 19:39:45 +10:00
Damien Elmes
a29bd7c9f0 unbury when rendering deck list
We were previously relying on the sched_timing_today() call in the
backend, but v3 doesn't call it, leading to cards remaining buried.
2021-10-16 21:11:09 +10:00
RumovZ
3672b0fe73
Switch CardInfoDialog to ts page (#1414)
* Only collect card stats on the backend ...

... instead of rendering an HTML string using askama.

* Add ts page Card Info

* Update test for new `col.card_stats()`

* Remove obsolete CardStats code

* Use new ts page in `CardInfoDialog`

* Align start and end instead of left and right

Curiously, `text-align: start` does not work for `th` tags if assigned
via classes.

* Adopt ts refactorings after rebase

#1405 and #1409

* Clean up `ts/card-info/BUILD.bazel`

* Port card info logic from Rust to TS

* Move repeated field to the top

https://github.com/ankitects/anki/pull/1414#discussion_r725402730

* Convert pseudo classes to interfaces

* CardInfoPage -> CardInfo

* Make revlog in card info optional

* Add legacy support for old card stats

* Check for undefined instead of falsy

* Make Revlog separate component

* drop askama dependency (dae)

* Fix nightmode for legacy card stats
2021-10-14 19:22:47 +10:00
Damien Elmes
ca32a46594 handle AnkiDroid's two element new intervals
https://github.com/ankidroid/Anki-Android/issues/8889
2021-10-07 23:15:51 +10:00
Damien Elmes
944b064e54 update Rust deps 2021-10-02 20:42:03 +10:00
Damien Elmes
8e56853c50 fix a clippy lint in latest Rust 2021-10-02 14:13:49 +10:00
Damien Elmes
0bb273a0ed replace the old stripHTML() methods with the backend implementation
Python's regex engine performs pathologically on regexes like
'<!--.*?-->' when fed a large string of repeating '<!--' clauses.
Thanks to JaimeSlome / security@huntr.dev for the report; closes #1380.

Solved by switching to the Rust implementation, which does not suffer
from this issue.

entsToText(), minimizeHTML(), and the old regex constants have been
removed; they do not appear to be used by any add-ons.
2021-10-01 23:15:45 +10:00
Damien Elmes
a174c41801
Merge pull request #1397 from RumovZ/column-tooltips
Add tooltips for some browser columns
2021-10-01 19:12:59 +10:00
Damien Elmes
3940b0587d avoid extra allocation in replacement 2021-10-01 14:43:38 +10:00
RumovZ
ee2ecd0700 Add tooltips for some browser columns 2021-09-30 13:15:09 +02:00
abdo
018594c5fb Strip isolation chars from filenames in Check Media screen 2021-09-27 22:15:37 +03:00
Damien Elmes
9daf037c0b
Merge pull request #1264 from hgiesel/tageditor2
Svelte Tag Editor v2
2021-09-15 20:21:16 +10:00
Damien Elmes
70a367137d handle filtered case when repositioning 2021-09-13 14:57:41 +10:00
Damien Elmes
6296e93c21 check for invalid collections in full_upload() 2021-09-13 11:50:13 +10:00
Damien Elmes
6fb6ffa1d7 tentative fix for learning count underflow
The 'avoid showing learning card twice' logic is now only applied
when the next learning card was already due to be shown. This'll mean
there will be cases where a learning card does get shown twice near
the end, but it makes the behaviour easier to reason about, for both
us and end users.
2021-09-13 11:08:55 +10:00
Damien Elmes
c9d3e5462e bikeshedding: amount -> match_limit
+ convert from u32 in backend method
2021-09-12 11:57:49 +10:00
Henrik Giesel
e85c93f3e7 Cap tag suggestions at 500 2021-09-10 01:13:50 +02:00
Damien Elmes
4da1c77220 add basic tag completion to backend
Matches should arrive in alphabetical order. Currently results are not
capped (JS should be able to handle ~1k tags without too much hassle),
and no reordering based on match location is done. Matches are substring
based, and multiple can be provided, eg "foo::bar" will match
"foof::baz::abbar".

This is not hooked up properly on the frontend at the moment -
updateSuggestions() seems to be missing the most recently typed character,
and is not updating the list of completions half the time.
2021-09-09 15:38:08 +02:00
Damien Elmes
a961013961 fix case of fields table 2021-09-07 10:22:22 +10:00
Damien Elmes
9a27e44ff8 don't panic when invalid note id provided to change notetype
https://forums.ankiweb.net/t/fatal-error-while-updating-deck/12743/2
2021-08-29 14:00:55 +10:00
Damien Elmes
dffa38c479 expose uncapped counts and total in tree node
In the future this information could be revealed on hover, to make it
clearer how limits are being applied.
2021-08-22 15:32:46 +10:00
Damien Elmes
05124eecf7 adding options to sort by ascending/descending ease 2021-08-22 15:32:46 +10:00
Damien Elmes
62223499c4 fix errors when undoing/redoing after a queue-invalidating operation
There were a few issues going on here:

- If some operation had invalidated the queues, they were subsequently
recreated with a call to .get_queues() in the undo handling code. This
could happen after the changes to the card had already been reverted,
leading to a queue state that didn't match our expectations.
- More generally, it's not safe to assume our mutations will apply
cleanly after the queue has been rebuilt. The next card will vary
depending on the number of remaining cards when interspersing cards of
different types, and a queue-invalidating operation will have changed
the learning cutoff.

So rather than rebuilding the queues on demand, we now check that they
already exist, and were created at the time we expect. If not, we
invalidate them and skip applying the mutations, and a subsequent
refresh of the UI should rebuild the queues correctly.

As part of this change, the cutoff snapshot has been moved into the
normal answer update object.

One possible downside here is that adding a note during review may cause
a newly due learning card to appear when undoing a different review.
If this proves to be a problem, we could potentially note down the
learning cutoff and apply it when queues are rebuilt later.
2021-08-22 15:32:46 +10:00
Damien Elmes
b9402b5c47 support limiting interday learning cards by review limit again
Context: https://forums.ankiweb.net/t/more-cards-today-question-about-v3/12400/10

Previously, interday learning cards and reviews were gathered at the
same time in v3, with the review limit being applied to both of them. The
order cards were gathered in would change the ratio of gathered learning
cards and reviews, but as they were displayed together in a single count,
a changing ratio was not apparent, and no special handling was required
by the deck tree code.

Showing interday learning cards in the learning count, while still
applying a review limit to them, makes things more complicated, as
a changing ratio will result in different counts. The deck tree code
is not able to know which order cards will appear in, so without changes,
we would have had a situation where the deck list may show different counts
to those seen when clicking on a deck.

One way to solve this would have been to introduce a separate limit for
interday learning cards. But this would have meant users needed to
juggle two different limits, instead of having a single one that controls
total number of (non-intraday) cards shown.

Instead, the scheduler now fetches interday cards prior to reviews -
the rationale for that order is that learning cards tend to be more
fragile/urgent than reviews. The option to show learning cards
before/after/mixed with reviews still exists, but it applies only after
cards have been capped to the daily limit.

To ensure the deck tree code matches the counts the scheduler gives,
it too applies limits to interday learning cards first, and reviews
afterwards.
2021-08-22 15:32:46 +10:00
Damien Elmes
6977b6ab6f fix new clippy warning 2021-08-22 15:32:46 +10:00
Damien Elmes
131c8b72f8 add options to v3 to preserve new card gather order
Allows cards to be presented in deck order when gather priority is set
to 'deck'.
2021-08-20 12:03:32 +10:00
Damien Elmes
53fe7e574e handle ampersand entities in image filenames
In the old HTML editor, filenames were % escaped before feeding them to
beautifulsoup, causing bare ampersands to be left alone. The new HTML
editor reads content from the DOM, where a bare ampersand has been
transformed into an &amp;, and that gets saved back into the field,
so the media check now needs to deal with it for images as well.

https://forums.ankiweb.net/t/causing-problems-with-image-names/12171
2021-08-19 23:43:40 +10:00
Damien Elmes
48c121e4f3 filtered decks w/ scheduling disabled in v3 now log reviews 2021-08-19 20:25:29 +10:00
Damien Elmes
8830d33826 revert some interday learning changes in v3
Interday learning cards are now counted in the learning count again,
and are no longer subject to the daily review limit.

The thinking behind the original change was that interday learning cards
are scheduled more like reviews, and counting them in the review count
would allow the learning count to focus on intraday learning - the red
number reflecting the fact that they are the most fragile memories. And
counting them together made it practical to apply the review limit
to both at once.

Since the release, there have been a number of users expecting to see
interday learning cards included in the learning count (the latest being
https://forums.ankiweb.net/t/feedback-and-a-feature-adjustment-request-for-2-1-45/12308),
and a good argument can be made for that too - they are, after all, listed
in the learning steps, and do tend to be harder than reviews. Short of
introducing another count to keep track of interday and intraday learning
separately, moving back to the old behaviour seems like the best move.

This also means it is not really practical to apply the review limit to
interday learning cards anymore, as the limit would be split between two
different numbers, and how much each number is capped would depend on
the order cards are introduced. The scheduler could figure this out, but
the deck list code does not know card order, and would need significant
changes to be able to produce numbers that matched the scheduler. And
even if we ignore implementation complexities, I think it would be more
difficult for users to reason about - the influence of the review limit
on new cards is confusing enough as it is.
2021-08-19 16:40:12 +10:00
Damien Elmes
272a610832 ensure interday learning cards update deck limits
https://forums.ankiweb.net/t/more-cards-today-question-about-v3/12400
2021-08-19 12:50:11 +10:00
Damien Elmes
d333d0da74 allow adjusting tag case in rename
https://forums.ankiweb.net/t/2-1-46-renaming-tags-to-tags-not-working/12426
2021-08-19 11:46:01 +10:00
Damien Elmes
1f4d54efda allow repositioning of new cards while suspended
https://forums.ankiweb.net/t/no-longer-can-reposition-a-suspended-card/12241
2021-08-15 15:06:25 +10:00
Damien Elmes
187944615e check both queues when popping answered card
If multiple answers are applied in succession, a newly-due learning card
may be in front of the next main entry.
2021-08-09 18:29:04 +10:00
abdo
52fac0557a Fix double quotes being escaped twice in dupe search
maybe_quote() already escapes double quotes.
2021-08-09 05:05:20 +03:00
Damien Elmes
6548ebb516 fall back on default deck in v3 queue gather as well
Missed in the previous change.

https://forums.ankiweb.net/t/error-when-trying-to-choose-deck-in-new-anki/11829/11
2021-08-09 11:04:37 +10:00
Damien Elmes
94913ec23f fallback on default deck in congrats screen
https://forums.ankiweb.net/t/error-when-trying-to-choose-deck-in-new-anki/11829
2021-08-04 12:57:46 +10:00
Damien Elmes
746b19e38c fix undoing last learning card corner case 2021-08-04 10:29:01 +10:00
Damien Elmes
7ba35b7249 support updating multiple notes in one transaction/undo op 2021-08-02 17:07:26 +10:00
Damien Elmes
88a3fd8d7b support updating multiple cards in one transaction/undo op 2021-08-02 16:59:02 +10:00
Damien Elmes
cc65196b0a revert to defaults when schema 11 deck config can not be read
Allows decks with missing/null values like ivlFct to be opened.

https://forums.ankiweb.net/t/i-cant-access-ankiweb/11814
2021-08-02 16:27:48 +10:00
Damien Elmes
070f57fcc5 don't hide learning count on congrats screen when learning is overdue
The v3 scheduler will delay the final card from being shown twice in
a row, but the overdue case was being treated the same as the no-learning
case, leading to the message being hidden.
2021-08-02 15:57:09 +10:00
Damien Elmes
3cec1a35dc update learning cutoff when counts are zero 2021-08-02 15:03:14 +10:00
Damien Elmes
94c7c3282b fix learning cutoff not updating during review 2021-08-02 15:03:14 +10:00
Damien Elmes
cf731c6bad fix v3 not honoring initial ease factor
Closes #1317
2021-07-31 14:57:04 +10:00
RumovZ
d090bd7e21 Use existing 'no such field' tr string 2021-07-29 08:30:49 +02:00
RumovZ
d6e5f3c67c Allow duplicate templates if {{Card}} is used 2021-07-28 22:32:38 +02:00
RumovZ
c0dd769090 Skip new notetype checks when importing apkg 2021-07-28 21:46:51 +02:00
RumovZ
be1b524396 Find template errors hidden by conditionals 2021-07-28 12:13:14 +02:00
RumovZ
4d7fcf585a Check for invalid conditionals on templates 2021-07-28 11:53:31 +02:00
Damien Elmes
6fe6570da3 fix reqwest build on Linux again 2021-07-24 10:18:09 +10:00
Damien Elmes
d73852f272 use separate integration test for links
If we run into issues with unreliable network connections in the future,
we'll be able to mark the test as flaky so Bazel can retry it multiple
times.
2021-07-24 10:12:25 +10:00
Damien Elmes
56545db339
Merge pull request #1308 from RumovZ/check-help
Build and check help links on the backend
2021-07-24 09:37:23 +10:00
RumovZ
07752f43cd Fix linkcheck result message 2021-07-23 19:40:51 +02:00
Damien Elmes
b1dedb1b1f disable link check outside CI 2021-07-23 20:22:32 +10:00
Damien Elmes
0578729468 be explicit about .html suffix 2021-07-23 20:22:32 +10:00
Damien Elmes
1ecda8cf4d fix clippy lints 2021-07-23 20:22:32 +10:00
Damien Elmes
379694915e add linkcheck to Bazel 2021-07-23 20:22:32 +10:00
RumovZ
503bdb8c22 Add check to ensure parsed URL equals link 2021-07-23 11:44:19 +02:00
RumovZ
6ef52726da Improve error message for unknown fragment 2021-07-23 10:32:29 +02:00
RumovZ
f2f19e8b45 Remove native HelpPage enum
Also remove oneof from pb enum and handle strs in Python.
2021-07-22 16:32:49 +02:00
RumovZ
55e1f178ef Add links.proto and backend module 2021-07-22 10:05:38 +02:00
RumovZ
80264c3b56 Add links.rs for checking and building links 2021-07-22 10:03:03 +02:00
Damien Elmes
5e10087aae handle notes with missing cards in browser
https://forums.ankiweb.net/t/2-1-45-release-candidate/11362/30
2021-07-22 14:58:57 +10:00
Damien Elmes
c944dd048e strip invalid Unicode chars in media check 2021-07-17 18:30:19 +10:00
Damien Elmes
2f434dd74d fix comment + copy/paste error 2021-07-17 09:02:14 +10:00
Aleksa Sarai
d7caafe91c
scheduler: use deck config's initial ease in set_due_date
Previously we would just use 250% ease for any new card that had no
pre-configured ease, but this will result in decks that have
non-standard ease values to have "set due date" cards in them that don't
match. In order to make this somewhat more efficient, we cache
deckid->ease lookups during this operation.

Ref: <https://forums.ankiweb.net/t/set-due-date-doesnt-obey-default-ease-factor/9184>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2021-07-16 12:33:15 +10:00
Damien Elmes
96e6e216a7
Merge pull request #1293 from RumovZ/find-and-replace
Find & Replace
2021-07-16 10:47:16 +10:00
Damien Elmes
bf507cca98 move from Python's URI escaping to IRI escaping in Rust
Should make non-Latin text readable in the HTML editor, without the
breakages reverted in the previous change.
2021-07-16 10:38:00 +10:00
RumovZ
a5193339e7 Rework Find & Replace dialog:
- Add option to affect whole collection
- Allow to open without selection
- Add parameter for presetting field
2021-07-13 16:33:45 +02:00
RumovZ
b148b7b5e0 Fix find & replace for notes without 'field_name'
Distinguish between no 'field_name' passed and 'field_name' not on note.
2021-07-13 16:21:13 +02:00
Damien Elmes
f649f6c92a minor tidyup in protobuf build script 2021-07-12 16:15:38 +10:00
Damien Elmes
9c354f5e6b drop the cloze-in-basic check for now
Unfortunately a popular note taking tool has been misusing cloze
markers in its deck exports. We may want to add this back in the
future, but we'll probably want to start by warning users, to give
people time to adjust.
2021-07-11 20:31:02 +10:00
Damien Elmes
0e7411188b configs.proto plural workaround no longer necessary 2021-07-11 19:35:18 +10:00
Damien Elmes
e61a611af7 rename Config in protobuf to avoid conflict with module name
+ use the enum directly, instead of wrapping it in an object

Python code retains the old "Config" name.
2021-07-11 19:27:08 +10:00
Damien Elmes
185e9acd22 split out remaining tags, stats, media and rendering 2021-07-10 23:16:18 +10:00
Damien Elmes
35b059ecdb split out sync, search, scheduler & config 2021-07-10 21:33:12 +10:00
Damien Elmes
9e0a295ab9 split out decks, deckconfig, notes, notetypes 2021-07-10 20:44:22 +10:00
Damien Elmes
18851ace47 split out cards and collection 2021-07-10 19:52:31 +10:00
Damien Elmes
616db33c0e refactor protobuf handling for split/import
In order to split backend.proto into a more manageable size, the protobuf
handling needed to be updated. This took more time than I would have
liked, as each language handles protobuf differently:

- The Python Protobuf code ignores "package" directives, and relies
solely on how the files are laid out on disk. While it would have been
nice to keep the generated files in a private subpackage, Protobuf gets
confused if the files are located in a location that does not match
their original .proto layout, so the old approach of storing them in
_backend/ will not work. They now clutter up pylib/anki instead. I'm
rather annoyed by that, but alternatives seem to be having to add an extra
level to the Protobuf path, making the other languages suffer, or trying
to hack around the issue by munging sys.modules.
- Protobufjs fails to expose packages if they don't start with a capital
letter, despite the fact that lowercase packages are the norm in most
languages :-( This required a patch to fix.
- Rust was the easiest, as Prost is relatively straightforward compared
to Google's tools.

The Protobuf files are now stored in /proto/anki, with a separate package
for each file. I've split backend.proto into a few files as a test, but
the majority of that work is still to come.

The Python Protobuf building is a bit of a hack at the moment, hard-coding
"proto" as the top level folder, but it seems to get the job done for now.

Also changed the workspace name, as there seems to be a number of Bazel
repos moving away from the more awkward reverse DNS naming style.
2021-07-10 19:17:05 +10:00
Damien Elmes
80b98e0db8 move protobuf into separate folder in preparation for multiple files 2021-07-09 21:02:40 +10:00
RumovZ
5067622751 Add pylib/browser.py for literal config keys
Also, remove config bools for sort order.
2021-07-05 12:44:48 +02:00
Henrik Giesel
aacf20531e Add NOTETYPE_FIELD sidebar items 2021-06-30 21:02:23 +02:00
Damien Elmes
79ec9b14b1 zero out graves before comparison, instead of at creation
Will allow us to turn the check back on in the future without a client
update
2021-06-30 10:08:52 +10:00
Damien Elmes
3ee18fb854 ignore graves in sanity check 2021-06-30 09:31:02 +10:00
Damien Elmes
576b141e2b add back in missing check for upload size
Compression now happens up-front, so we can tell in advance if the
upload size has been exceeded.
2021-06-29 10:50:03 +10:00
Damien Elmes
e3c9808b79 catch invalid card ordinals in DB check
https://forums.ankiweb.net/t/error-corrupted-note/10976
2021-06-29 10:50:03 +10:00
Damien Elmes
2a93355824 PEP8 cards.py 2021-06-27 12:12:23 +10:00
Damien Elmes
ca0374782e update to latest rusqlite 2021-06-25 16:22:21 +10:00
Damien Elmes
59e17950ad update most rust deps; skip rusqlite 2021-06-25 15:35:25 +10:00
Damien Elmes
73d9391f64 update undo skipping; exclude deck/tag expand/collapse
Instead of calling a method inside the transaction body, routines
can now pass Op::SkipUndo if they wish the changes to be discarded
at the end of the transaction. The advantage of doing it this way is
that the list of changes can still be returned, allowing the sync
indicator to update immediately.

Closes #1252
2021-06-25 09:16:15 +10:00
Damien Elmes
b693bdbf28 cope with collections where latexsvg is not a bool 2021-06-24 15:29:21 +10:00
Damien Elmes
47284867f9 report object type where json decode fails 2021-06-24 15:28:24 +10:00
Damien Elmes
0b697f5161 expose scheduler js option in deck config 2021-06-24 14:38:10 +10:00
Damien Elmes
114eec6585 allow customization of default search text
https://forums.ankiweb.net/t/whats-the-default-search-when-browsing-now-also-how-do-i-choose-to-browse-current-deck/8922
2021-06-24 11:24:30 +10:00
Damien Elmes
b392020798 fix clippy lints for latest Rust 2021-06-21 13:09:36 +10:00
Damien Elmes
c79f8ba88f in/out -> request/response
The saved characters weren't worth the increased difficulty when
reading, and the fact that we were deviating from protobuf norms.
2021-06-20 15:49:20 +10:00
Damien Elmes
a58826941f add rustls-native-certs
closes #1241
2021-06-18 18:15:41 +10:00
Damien Elmes
2e53dc63c8
Merge pull request #1230 from RumovZ/fields-check
Check for misplaced or missing clozes when adding and in the editor
2021-06-17 21:26:16 +10:00
RumovZ
87a50f22e6 Assign dupe error the lowest precedence 2021-06-16 11:42:40 +02:00
Henrik Giesel
0f658de702 Add escape_anki_wildcards_for_search_node 2021-06-16 09:25:27 +02:00
Henrik Giesel
b8142d33c8 Revert "Remove special treatment of tag:none"
This reverts commit 95285ef66d02d9d3f0d560f70514b16e8a2f7bf5.
2021-06-16 17:19:21 +10:00
Henrik Giesel
b19e07c135 Remove special treatment of tag:none 2021-06-16 17:19:21 +10:00
Henrik Giesel
c5faf39d7c Make Browser root nodes use "_*" uniformly 2021-06-16 17:19:21 +10:00
Damien Elmes
ecdd668f9a update Rust deps 2021-06-16 16:10:57 +10:00
RumovZ
d7340d3f07
Fix missing hyphen escape when normalizing search (#1233)
* Fix missing hyphen escape when normalizing search

* Add some more tests for normalization

* Use char to satisfy clippy
2021-06-15 10:02:39 +10:00
Damien Elmes
220fca9a1d handle <br/> when rendering a single line
+ case-insensitive matching

https://forums.ankiweb.net/t/html-editor-modifies-note-when-a-field-with-break-tags-is-opened/10772
2021-06-14 13:05:48 +10:00
RumovZ
aeedb4dc11 Add check for out-of-place/missing clozes 2021-06-12 10:02:21 +02:00
Damien Elmes
61e86cc29d new change notetype implementation for the frontend
- changes can now be undone
- the same field can now be mapped to multiple target fields, allowing
fields to be cloned
- the old Qt dialog has been removed
- the old col.models.change() API calls the new code, to avoid
breaking existing consumers. It requires the field map to always
be passed in, but that appears to have been the common case.
- closes #1175
2021-06-10 22:19:24 +10:00
Damien Elmes
40dc2e217c fix spurious text in header injected by rust analyzer 2021-06-10 09:06:37 +10:00
Damien Elmes
1f2567e04c add notetype changing to backend 2021-06-09 20:56:52 +10:00
Damien Elmes
410660990e add LIFO sorting options for new cards 2021-06-08 14:01:46 +10:00
Damien Elmes
1479957538 fix note changes triggering a queue rebuild 2021-06-08 12:09:35 +10:00
Damien Elmes
4325f867a7
Merge pull request #1217 from RumovZ/flag-ftl
Mark search-invalid-flag as a new string
2021-06-03 10:50:11 +10:00
RumovZ
f33bd4ca11 search-invalid-flag -> search-invalid-flag-2 2021-06-02 21:49:33 +02:00
Damien Elmes
2903848c62 handle duplicate ids in values when upgrading deck config
Multiple configs with the same inner id would lead to errors like the
following when trying to open the collection:

DeckConfigInner.interval_multiplier: invalid wire type: StartGroup (expected ThirtyTwoBit)
2021-06-02 17:12:59 +10:00
Damien Elmes
578b1b0552
Merge pull request #1213 from RumovZ/new-flags
Add pink, turquoise and purple flags
2021-06-02 11:22:26 +10:00
RumovZ
5e7a47fdc7 Rename violet to pink 2021-06-01 18:14:43 +02:00
Damien Elmes
50961a9196 push review randomizing into SQL
This makes the review backlog case more expensive, since we end up
shuffling items outside the daily limit, but for the common case it's
about the same speed, and it means we don't need two separate sorting
steps. New cards remain handled the same way, since a backlog
is common there.

Also ensures that interday learning cards honor the deck sorting, and
that the non-default sort orders shuffle at the end.
2021-06-01 14:50:35 +10:00
Damien Elmes
562787bce1 add options to sort reviews by deck
https://forums.ankiweb.net/t/is-studying-subdeck-by-subdeck-broken-in-2-1-44-mac/10458/2
2021-06-01 13:22:39 +10:00
Damien Elmes
775beda6ce fix subdeck order not being preserved by active_decks in v3 2021-06-01 11:40:25 +10:00
RumovZ
c97c6c6e98 Add violet, turquoise and purple flags 2021-05-31 12:03:30 +02:00
Damien Elmes
29c4869aef remove deck protobuf from frontend
Like the previous change, avoid exposing the protobuf as a public API
for now. It requires more thought, and is probably better done with
either extra helper accessors like decks.name(), or via a native class.
2021-05-31 16:31:06 +10:00
Damien Elmes
25e4e4c8f6 fix exporting of non-default deck configs 2021-05-31 16:27:58 +10:00
Damien Elmes
93459cc48f
Merge pull request #1200 from RumovZ/template-checks
Template checks
2021-05-29 10:28:06 +10:00
RumovZ
44506136e2 Use HashSet for special fields 2021-05-28 15:42:09 +02:00
RumovZ
329f89c093 Add const for special fields and doc 2021-05-28 11:58:46 +02:00
RumovZ
33bf391114 Allow empty field name in templates 2021-05-28 11:37:05 +02:00
RumovZ
2e923db6bd Add checks for parsed templates
Combine existing check for unparsable templates with a check for unknown
field names and a check for front sides without any field replacement.
Updating the notetype's fields now mutates the parsed templates, so the
checks can run on the final templates.
2021-05-28 10:07:31 +02:00
Damien Elmes
e3db8641d1 slightly simplify transact_inner() 2021-05-28 14:45:36 +10:00
Damien Elmes
af50c445dd don't depend on timer to increment between ops in unit test 2021-05-28 11:43:24 +10:00
Damien Elmes
aa7d2721c9 avoid bumping mtime when nothing has changed
+ update sync indicator after every op
+ skip mtime bump on undo/redo
2021-05-28 11:09:16 +10:00
Damien Elmes
6cc713cbe8 add v3 scheduler to prefs screen 2021-05-27 23:09:49 +10:00
RumovZ
c61587b1de Use HashMap in identical template check 2021-05-27 12:45:17 +02:00
RumovZ
3dfa1de68b Check for clozes when saving notetype
Error if:
- Cloze notetype lacks a cloze field on either template side.
- Non-cloze notetype has a cloze field on any template.
2021-05-27 12:01:05 +02:00
Damien Elmes
57ec4cc7b5 change get_queued_cards() to no longer return congrats info 2021-05-26 12:59:45 +10:00
RumovZ
6fae0ea21f Update tests to avoid duplicate front templates 2021-05-25 21:58:12 +02:00
RumovZ
60131eab23 Check for identical templates before saving 2021-05-25 21:57:49 +02:00
RumovZ
fa19f590e8 Add details to TemplateSaveError 2021-05-25 21:01:03 +02:00
RumovZ
bd8c13067d Escape HTML in template error message 2021-05-25 19:40:15 +02:00
Damien Elmes
e9309c5378 expose the ability to get/set aux notetype/template keys
template keys are not currently adjusted when card templates are
repositioned.
2021-05-25 22:13:53 +10:00
Damien Elmes
050ef11a96 pass css and latex svg flag back from rendering op
This could potentially help us avoid having to refetch the notetype
during study in the future, though updates to Note initialization and
the LaTeX handling would be required first.
2021-05-25 18:41:43 +10:00
Damien Elmes
02c7f7989e support passing in a native notetype object to render_uncommitted_card() 2021-05-25 16:58:06 +10:00
Damien Elmes
52676f1555 fix a panic when browser encounters a filtered card outside filtered deck
the -99999 due date overflows the i32, yielding to a stuck interface
when running in a debug build
2021-05-25 13:44:42 +10:00
Damien Elmes
adcdb422c5 config updates by the frontend now skip undo by default 2021-05-24 14:50:46 +10:00
Damien Elmes
ea20c31b53 suppress some tests around the daily rollover 2021-05-24 14:18:07 +10:00
Damien Elmes
29f9717c84 fix new ease not being applied to card on lapse
+ zero remaining steps when graduating (they shouldn't have been doing
any harm, but this is neater)
+ add some more tests that cover these cases
2021-05-24 10:04:56 +10:00
Damien Elmes
3d4cf26758 expose undoable config changes to frontend; refresh sidebar
The browser header handling still needs updating
2021-05-21 17:50:41 +10:00
Damien Elmes
99b7da49a9 report changed cards when changing deck/flag
+ fix repeated flag shortcut not toggling
2021-05-21 16:03:05 +10:00
Damien Elmes
9aba412b44
Merge pull request #1180 from RumovZ/say-blank
Pronounce "[...]" as "blank" with TTS
2021-05-20 18:48:10 +10:00
RumovZ
b5c29fb498 Pronounce "[...]" as "blank" with TTS 2021-05-20 09:42:38 +02:00
Damien Elmes
b412bf97fb add a separate DeckId search for decks with children
- The "unbury deck" option was broken, as it was ignoring child
decks. It would be nice if we could use active_decks instead, but
plugging that into the old scheduler without breaking undo seems a bit
tricky.
- Remove the implicit From impl for decks, so we need to be forced to
think about whether we want child decks or not.
2021-05-20 11:44:37 +10:00
Damien Elmes
581480191a show note about old options screen if any add-ons are installed 2021-05-19 16:20:25 +10:00
Damien Elmes
1f77be01e7 fix accidental schema bump in undo that was forcing full sync 2021-05-19 14:17:49 +10:00
RumovZ
c1066167fa Adjust search syntax for filtered deck presets 2021-05-17 12:14:02 +02:00
Damien Elmes
1f16ce2096 shift learning fuzz into answering stage in test scheduler
When shown on the answer buttons, it's too distracting
2021-05-17 13:05:42 +10:00
RumovZ
92aa5404c9 Preserve formatting of regex error 2021-05-16 21:07:35 +02:00
RumovZ
9b2cd52b97 Remove SearchErrorKind::Regex as it's never used 2021-05-16 21:07:03 +02:00
Damien Elmes
13519a929c rework various aspects of the test scheduler
- Daily limits are no longer inherited - each deck limits its own
cards, and the selected deck enforces a maximum limit.
- Fetching of review cards now uses a single query, and sorts in advance.
In collections with a large number of overdue cards and decks, this is
faster than iterating over each deck in turn.
- Include interday learning count in review count & review limit, and
allow them to be buried.
- Warn when parent review limit is lower than child deck in deck options.
- Cap the new card limit to the review limit.
- Add option to control whether new card fetching short-circuits.
2021-05-16 20:23:07 +10:00
Damien Elmes
35063316d3 remove some duplicate code & add deck.or() helper 2021-05-14 22:35:52 +10:00
Damien Elmes
390a8421aa fix test scheduler undo + implement look-ahead
Instead of using a separate undo queue, the code now defers checking for
newly-due learning cards until the answering stage, and logs the updated
cutoff time as an undoable change, so that any newly-due learning cards
won't appear instead of a new/review card that was just undone.

Queue redo now uses a similar approach to undo, instead of rebuilding the
queues.
2021-05-14 22:16:53 +10:00
Damien Elmes
9990a10161 drop binary heap in test scheduler
The original rationale was avoiding a possible O(n) insertion if
the learning card was due outside the cutoff, but the increased code
complexity doesn't seem worth it, given that learning cards will
rarely grow above 1000.

Also added a currently-disabled test that demonstrates the current undo
handling behaviour is yielding incorrect counts; that will be reworked
in the next commit, and this change will make that easier.
2021-05-14 16:19:46 +10:00
Damien Elmes
dbbcb3e38c expose new sorting options in test scheduler options; move things around 2021-05-13 15:23:16 +10:00
Damien Elmes
a1bd6b481d pass sort options into test scheduler
- split new card fetch order and subsequent sort order; use latter
when building queues
- default to spacing siblings when burying is off, with options to
show each sibling in turn, and shuffle the fetched cards
2021-05-13 15:21:20 +10:00
Damien Elmes
b64f7a9456 fix burying in test scheduler
The bury new/review flags are now pulled from each card's home deck,
instead of using a global setting that had not been hooked up. This
unfortunately means we need to fetch the map of all decks up front, as
we need to be able to look up a deck configuration for cards that are
in filtered decks.

Fixes a "card was modified" error caused by cards being buried during
review, when they weren't removed up-front.
2021-05-12 12:00:15 +10:00
Damien Elmes
e737eb3088 is_stale() doesn't need to be passed deck
Deck changes will trigger a queue rebuild via requires_study_queue_rebuild()
2021-05-12 09:44:10 +10:00
Damien Elmes
49a1580566 use new API for test scheduler
Avoids duplicate work, and is a step towards allowing the next
states to be modified by third-party code.

Also:

- fixed incorrect underlined count, due to reviews being labeled as
learning cards
- fixed reviewer not refreshing when undoing a test review, by splitting
up backend queue rebuilding from frontend reviewer refresh
- moved answering into a CollectionOp
2021-05-11 13:06:03 +10:00
Damien Elmes
6775002709 better leech tag handling for test scheduler 2021-05-10 14:58:04 +10:00
Damien Elmes
3736e63a57 expose step counter and undone op changes in hook 2021-05-08 17:51:36 +10:00
Damien Elmes
e9e1edc64d move action names out of undo.ftl into actions.ftl 2021-05-08 17:11:54 +10:00
Damien Elmes
dfba66faa9 update Rust deps 2021-05-07 18:22:27 +10:00
Damien Elmes
d878de54c0 default to the v2 scheduler in new collections
ported from 53d9433d94
2021-05-06 19:09:28 +10:00
Damien Elmes
be994f4102 add support for custom undo steps, and merging multiple actions
Allows add-on authors to define their own label for a group of undoable
operations. For example:

def mark_and_bury(
    *,
    parent: QWidget,
    card_id: CardId,
) -> CollectionOp[OpChanges]:
    def op(col: Collection) -> OpChanges:
        target = col.add_custom_undo_entry("Mark and Bury")
        col.sched.bury_cards([card_id])
        card = col.get_card(card_id)
        col.tags.bulk_add(note_ids=[card.nid], tags="marked")
        return col.merge_undo_entries(target)

    return CollectionOp(parent, op)

The .add_custom_undo_entry() is for adding your own custom actions.
When extending a standard Anki action, instead store `target = 
col.undo_status().last_step` after executing the standard operation.

This started out as a bigger refactor that required a separate
.commit_undoable() call to be run after each operation, instead of
having each operation return changes directly. But that proved to be
somewhat cumbersome in unit tests, and ran the risk of unexpected
behaviour if the caller invoked an operation without remembering to
finalize it.
2021-05-06 16:39:06 +10:00
Damien Elmes
0fb745cdb9 add a valid, empty file so the check action works in Rust Analyzer 2021-05-05 15:53:27 +10:00
Damien Elmes
abab4826bb support undo for (renamed) unbury_deck() action 2021-04-30 20:03:20 +10:00
Damien Elmes
d0f3007fad remove some unneeded transactions from tests 2021-04-30 19:18:02 +10:00
Damien Elmes
449e3cc79d fix invalid outer transaction in clear_unused_tags() 2021-04-30 19:09:02 +10:00
Damien Elmes
cfb2a382b1 empty/restore media trash shouldn't clear undo 2021-04-30 19:07:30 +10:00
Damien Elmes
9a46ad6352 undoing of notetype fields
- fix stale cache issue
- update add cards screen in response to op changes
2021-04-30 17:15:59 +10:00
Damien Elmes
ea758f0092 update GUI to allow notetype addition undo
- backend now updates current notetype as part of addition
- frontend no longer implicitly adds, so we can assign a new name and
add in a single operation
2021-04-30 15:58:08 +10:00
Damien Elmes
2ff8c20686 update backend to support undoing of notetype changes 2021-04-30 12:54:59 +10:00
Damien Elmes
9d604f1ad0 make it more ergonomic to search directly via nodes in Rust 2021-04-30 11:37:55 +10:00
Damien Elmes
2902b64e82 move note/card removal for notetype out of storage layer 2021-04-29 19:44:09 +10:00
Damien Elmes
57eab0e33d make config prefix removal undoable 2021-04-29 19:16:02 +10:00
Damien Elmes
610f0b5254 use aux var when retrieving last deck for notetype
The adding case was already covered by defaults_for_adding(), but we
the code was using the old variable when adding new cards to an existing
note.
2021-04-29 19:15:27 +10:00
Damien Elmes
b160c2d85c move pub functions to top of notetype/mod.rs 2021-04-29 15:32:59 +10:00
Damien Elmes
86cf8949ea move pub functions to top of notes/mod.rs 2021-04-29 14:08:15 +10:00
Damien Elmes
07b5c35e90 confirm deletion is requiring a full sync 2021-04-28 22:21:16 +10:00
Damien Elmes
b22e851137 add some unit tests to deck config updating 2021-04-28 22:00:39 +10:00
Damien Elmes
a95cbb8515 DeckConfId -> DeckConfigId 2021-04-28 21:09:26 +10:00
Damien Elmes
1df86b28e8 sort deck on config update; fix id not being updated after deletion 2021-04-28 21:08:09 +10:00
Damien Elmes
92e4de16cf report pinned package issues upstream 2021-04-27 23:11:22 +10:00
Damien Elmes
cf2613e869 update Rust deps
- tokio 1.0
- updated reqwest, thanks to Rumo
- other minor dep updates

the reqwest build file has been split into two, as it was awkward
to manually update the combined file, and the platform gate is now
on the target in rslib/
2021-04-27 22:18:12 +10:00
Damien Elmes
f6a1a35113 update translations 2021-04-25 22:06:05 +10:00
Damien Elmes
30f5269304 hook new deck config screen up behind an env var 2021-04-22 10:59:16 +10:00
Damien Elmes
fd81700679 deckconf -> deckconfig 2021-04-20 21:54:24 +10:00
Damien Elmes
b448afdc57 change detection needs to ignore collection mtime bump 2021-04-20 19:52:47 +10:00
Damien Elmes
42d5d8071d check if deck changed before saving 2021-04-20 19:50:31 +10:00
Damien Elmes
4d1cedc8b2 implement deck config updating 2021-04-20 14:57:44 +10:00
Damien Elmes
4d4603c078 remove some imports duplicated by prelude 2021-04-19 18:22:43 +10:00
Damien Elmes
2cc23ce2bc
Merge pull request #1138 from RumovZ/introduced
Add search keyword for "first review in x days"
2021-04-19 18:22:15 +10:00
RumovZ
cac507b979 Use arg name instead of repeating it in format!() 2021-04-19 08:58:33 +02:00
RumovZ
194d251004 Use timestamp adding for writing cutoff 2021-04-19 08:44:13 +02:00
RumovZ
11cd1d9a26 Add sql condition for speedup in write_introduced 2021-04-19 08:43:32 +02:00
RumovZ
3cecc7157b Add sidebar filter "first review today" 2021-04-18 13:27:10 +02:00
RumovZ
e274ff26ab Add sqlwriter test for introduced 2021-04-18 12:32:02 +02:00
RumovZ
699437df0f Add cutoff_in_secs_from_days() helper method 2021-04-18 12:25:44 +02:00
RumovZ
5ec2e11de3 Add search keyword introduced
Filters for cards that had their first review within the last x days.
2021-04-18 12:14:18 +02:00
Damien Elmes
64ebc32b3d tidy up Rust imports
rustfmt can do this automatically, but only when run with a nightly
toolchain, so it needs to be manually done for now - see rslib/rusfmt.toml
2021-04-18 18:38:54 +10:00
Damien Elmes
262b50445c start on making deck config and schema/mod changes undoable
+ move timestamps into a struct in a separate file for convenience
2021-04-18 17:33:12 +10:00
Damien Elmes
76eb119870 add schema change prompt to removal, tweak return struct 2021-04-18 17:33:12 +10:00
RumovZ
fece39ca74 Maybe normalize name when preparing deck update 2021-04-18 09:16:43 +02:00
RumovZ
5e3e194289 Remove redundant imports 2021-04-18 08:43:46 +02:00
Damien Elmes
e71f7714ad as_str() -> as_native_str() 2021-04-18 09:33:39 +10:00
Damien Elmes
1acc679e8f hide NativeName inner value, and require explicit accessors 2021-04-18 09:29:35 +10:00
Damien Elmes
f924539574 create_missing_parents() can take a native name directly 2021-04-18 09:20:23 +10:00
RumovZ
20bd207f00 Give deck.name the newtype NativeDeckName
The deck name must be constructed by calling associated functions of
NativeDeckName, unless the name is guaranteed to be valid machine
name (like "Default").
NativeDeckName exposes methods to mutate the deck name and return
the human name.
The storage routines take &strs, but those should be slices of
NativeDeckNames to ensure machine form and normalization.
2021-04-17 22:47:04 +02:00
Damien Elmes
ac46d40db3
Merge pull request #1132 from RumovZ/refactor-decks
Refactor decks/mod.rs
2021-04-17 18:21:56 +10:00
Damien Elmes
8cf8c901fe fix parent limit handling
We can't calculate it on the backend, as adjusting a config may alter
the parent limit.

Also fix hidden deck name and missing separator.
2021-04-17 12:53:59 +10:00
RumovZ
c03acf832b Split Col impls in decks in pub and private blocks 2021-04-16 08:30:16 +02:00
RumovZ
5931631d76 Remove some imports in decks which are in prelude 2021-04-15 20:07:16 +02:00
RumovZ
3138fccaca Create decks/stats.rs 2021-04-15 20:06:16 +02:00
RumovZ
d2337e4cd3 Move some methods into decks/counts.rs 2021-04-15 19:53:11 +02:00
RumovZ
9dac8e2b5f Tidy up blocks and imports in decks/mod.rs 2021-04-15 19:43:35 +02:00
RumovZ
a1617760d4 Create decks/remove.rs 2021-04-15 19:35:54 +02:00
RumovZ
7225b7e4dc Create decks/add.rs 2021-04-15 19:29:52 +02:00
RumovZ
3f36db4f81 Create decks/reparent.rs 2021-04-15 19:17:43 +02:00
RumovZ
9c3671cf3f Create decks/name.rs 2021-04-15 18:46:01 +02:00
Damien Elmes
2953a821e5 don't let manual rescheduling affect card stats average
https://forums.ankiweb.net/t/set-due-date-has-a-weird-effect-on-cards-table/9289/2
2021-04-15 18:51:39 +10:00
Damien Elmes
7f738c11a2 deck config prototype work in progress
Still in the early stages, and not hooked up yet.
2021-04-14 22:33:10 +10:00
Damien Elmes
14ce1cccbb
Merge pull request #1126 from RumovZ/normalising-tweaks
Drop ANDs and optional double quotes when writing searches
2021-04-14 19:15:52 +10:00
Damien Elmes
c49d6ce49f run black/isort on Python scripts 2021-04-14 18:22:02 +10:00
RumovZ
2554e80ce8 Drop ANDs and optional quotes when normalising 2021-04-14 09:53:45 +02:00
Damien Elmes
28fdbd67ae add missing copyright headers to *.rs 2021-04-13 18:59:16 +10:00
Damien Elmes
dac990e4c2 fix misnamed timer property in deck config
The old JSON stored it as an int, but it's used as a boolean
2021-04-13 14:04:33 +10:00
RumovZ
fadec3dc5b Merge branch 'master' into backend-columns 2021-04-11 11:18:15 +02:00
RumovZ
801f52df40 Remove from_config variant in pb SortOrder
Instead, fetch the config order on the frontend and pass a builtin
variant into the backend.
That makes the following unnecessary:
* Resolving the config sort in search/mod.rs
* Deserializing the Column enum
* Config accessors for the sort columns
2021-04-10 11:13:42 +02:00
RumovZ
5982a777aa Rename SearchItems to ReturnItemtype 2021-04-10 10:14:41 +02:00
RumovZ
331df75b72 Remove internal clones of pb BrowserRow structs 2021-04-10 09:49:29 +02:00
RumovZ
48b70873cb Rename column label fields 2021-04-10 09:14:20 +02:00
RumovZ
b723159b3b Remove unused Serialize 2021-04-10 09:13:48 +02:00
RumovZ
d7f7deafd4 Store active browser columns in col state 2021-04-09 22:53:02 +02:00
RumovZ
769b5ac833 Remove superfluous muts 2021-04-09 22:51:18 +02:00
RumovZ
f04ea5a2c7 Move Column logic into main rslib 2021-04-09 19:09:48 +02:00
RumovZ
055a5e8a04 Remove pb SortKind enum and use pb Columns instead 2021-04-09 18:50:30 +02:00
RumovZ
bdd257e140 Merge SortKind enum into Column enum 2021-04-09 18:03:29 +02:00
RumovZ
c74078ea9e Unify state columns
* Remove duplicate backend columns
* Remove duplicate column routines
* Move columns on frontend from state to model
* Generate available columns from Colum enum
* Add second column label for notes mode
2021-04-08 23:48:24 +02:00
RumovZ
dd56dc6650 Rename columns for future mode-independent use 2021-04-08 23:43:48 +02:00
RumovZ
7a0cb08ac2 Merge browser row str methods 2021-04-08 20:45:47 +02:00
RumovZ
4692a48ef3 Add card mod column for notes mode 2021-04-08 20:14:10 +02:00
RumovZ
2350cd6e91 Add deck column for notes mode 2021-04-08 19:46:06 +02:00
RumovZ
c6ebb9b441 Merge row contexts 2021-04-08 13:51:46 +02:00
RumovZ
8a131da9a2 Add enum for column sorting 2021-04-08 11:40:24 +02:00
RumovZ
d8a0aa922c Add enum for column alignment 2021-04-08 11:28:29 +02:00
RumovZ
6c3c479906 Move BrowserColumn into BrowserColumns message 2021-04-08 10:16:06 +02:00
RumovZ
ee0ad6f5d6 Fix deck column serialization string 2021-04-06 23:03:30 +02:00
RumovZ
11bdeb9ca4 Add column logic on backend 2021-04-06 19:46:12 +02:00
RumovZ
929b7dc15b Make Column a strum 2021-04-06 16:54:09 +02:00
Damien Elmes
6e954e82a5 current deck change is now undoable
- make sure we set flag in changes when config var changed
- move current deck get/set into backend
- set_config() now returns a bool indicating whether a change was
made, so other operations can be gated off it
- active decks generation is deferred until sched.reset()
2021-04-06 21:52:06 +10:00
Damien Elmes
5676ad5101 update find&replace, and remove perform_op() 2021-04-06 17:07:38 +10:00
Damien Elmes
84fe309583 update scheduling ops
- migrate to CollectionOp()
- return actual change count when suspending/burying
- add helper to convert vec to vec of newtype
2021-04-06 16:38:42 +10:00
Damien Elmes
2de8cc1a94 update note ops
remove_note() now returns the count of removed cards, allowing us
to unify the tooltip between browser and review screen

I've left the old translation in - we'll need to write a script at
one point that gathers all references to translations in the code,
and shows ones that are unused.
2021-04-06 14:56:36 +10:00
Damien Elmes
3f62f54f14 more perform_op() tweaks
- pass the handler directly
- reviewer special-cases for flags and notes are now applied at
call site
- drop the kind attribute on OpChanges which is not needed
2021-04-06 10:14:11 +10:00
Damien Elmes
097121424b cache scheduling info
Saves us having to recalculate it for each browser row
2021-04-05 17:09:53 +10:00
Damien Elmes
ebf7cc61d4 switch next_day_at to a newtype 2021-04-05 16:17:26 +10:00
Damien Elmes
8d6b9d15a5 avoid fetching decks for each row
Like notetypes, there is a col.get_deck() routine which caches
fetches, so that successive fetches are cheap. This makes it simpler
to just fetch the deck at the start.

We were also attempting to fetch a deck with id 0 for each row; I've
changed this so that we only fetch it if the id is non-zero.

I18n uses an Arc internally, so it is cheap to clone. This allow us
to drop the lifetime specifiers on the context structures.
2021-04-05 15:13:32 +10:00
Damien Elmes
a18bb2af12 add booleans for various screens to OpChanges
The backend knows exactly which op has executed, and it saves us having
to re-implement this logic on each client.

Fixes the browser table refreshing when toggling decks.
2021-04-05 14:28:56 +10:00
Damien Elmes
3adf03f9cb add a unit test for multiple mutations 2021-04-05 11:52:23 +10:00
Damien Elmes
996d9f9bbc undo support for tag collapse; expand->collapse for consistency w/ decks 2021-04-05 11:47:12 +10:00
Damien Elmes
2168dfe63d add routine to set deck collapse state
Updating a deck via protobuf is now exposed on the backend, but not
currently on the frontend - I suspect we'll be better off writing
separate routines for the actions we need instead, and we get a better
undo description for free.

This is currently causing an ugly redraw in the browse screen, which
will need fixing.
2021-04-05 11:19:04 +10:00
Damien Elmes
42a4d11416 embed deck config and expose to frontend 2021-04-04 22:52:53 +10:00
Damien Elmes
037df9522b embed notetype messages 2021-04-04 21:57:17 +10:00
Damien Elmes
c4b3ab62c8 embed deck messages 2021-04-04 21:41:16 +10:00
Damien Elmes
1a4c4373d2 expose read-only access to new notetype objects 2021-04-04 20:45:37 +10:00
Damien Elmes
c60b88cd2f expose read-only access to new deck objects 2021-04-04 20:39:56 +10:00