Commit Graph

1376 Commits

Author SHA1 Message Date
Damien Elmes
a39a3b4d34 Update to latest rules_rust and Rust 1.64 2022-09-24 11:12:58 +10:00
Damien Elmes
b97a8bfa26 Fix daily counts being included in apkg import
Fixes https://github.com/ankidroid/Anki-Android/issues/12477
2022-09-24 09:05:29 +10:00
Damien Elmes
09cb8b3cf6 Always scan for media changes
The check runs in the background, and the dentry cache and SQLite
page cache should make this more efficient after the first run.

Closes #1736
2022-09-20 16:13:10 +10:00
RumovZ
e7af0febb1
More template checks (#2032)
* Show warning if multiple type boxes are used

* Report templates referencing media in Media Check

* Apply suggestions from code review

* Fix media-check.ftl

* Only report media references with fields

Like `<img src={{Front}}>`.
Also report Anki sound tags and latex.

* Loop existing media regexes
2022-09-05 16:52:25 +10:00
RumovZ
e39fb74e82
Enable state-dependent custom scheduling data (#2049)
* Enable state-dependent custom scheduling data

* Next(Card)States -> SchedulingStates

The fact that `current` was included in `next` always bothered me,
and custom data is part of the card state, so that was a bit confusing
too.

* Store custom_data in SchedulingState

* Make custom_data optional when answering

Avoids having to send it 4 extra times to the frontend, and avoids the
legacy answerCard() API clobbering the stored data.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-09-05 16:48:01 +10:00
Damien Elmes
0bcb3a3564 Add non-legacy backend interface for adding new decks 2022-09-04 14:12:29 +10:00
RumovZ
1f8189fe91
Some import/export features and fixes (#2038)
* Keep filtered decks when importing apkg

If all original decks exist and scheduling is included.

* Create missing decks from csv

* Export original decks if with_scheduling

* Also remap original deck ids on import

* Update imported filtered decks

* Fix meta column being mapped to tags

* Fix ids in csv deck and notetype columns

Note: This implies names which parse to an i64 will be seen as ids,
likely resulting in the intended deck/notetype not being found.

* Check for scheduling with revlog and deck configs

Might help with cases in which scheduling was included, but all cards
are new. In such a case, filtered deck should not be converted.

* Fix duplicate with same GUID being created

* Remove redundant `distinct`s from sql query

* Match notes by _either_ guid _or_ first field

* Refactor to emphasise GUID/first field distinction

* Export default deck and config if with scheduling

* Fix default deck being exported if it's a parent
2022-09-03 11:29:06 +10:00
RumovZ
31b7464c67
Add card meta for persisting custom scheduling state (#2040)
* Add card meta for persisting custom scheduling state

* Rename meta -> custom_data

* Enforce limits on size of custom data

Large values will slow down table scans of the cards table, and it's
easier to be strict now and possibly relax things in the future than
the opposite.

* Pack card states and customData into a single message

+ default customData to empty if it can't be parsed

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-09-02 11:22:49 +10:00
Matthias Metelka
d110c4916c
Introduce setting to collapse field by default (#1990)
* Introduce setting to collapse field by default

* Fix schema order

* Change wording from adjective to imperative

sounds a bit less clunky

* Update rslib/src/notetype/schema11.rs (dae)

* Keep settings in single column

* Add back Toggle Visual Editor string

* Add RichTextBadge component and show it conditionally

* Reverse input order depending on default setting

* Make PlainTextInput border-radius responsive to toggle states

* Prevent first Collapsible transition differently

* Focus inputs after Collapsible transition

The double tick calls are just a temporary solution until I find the exact moment an input is focusable again.

* Use requestAnimationFrame to await focusable state

Note: Svelte tick doesn't seem to work in this scenario.
2022-08-31 23:34:39 +10:00
RumovZ
79fbb6c8d8
Keep content of unmapped fields when importing (#2023)
* Keep content of unmapped fields when importing

* Test new behaviour

* Fix typo in `canonify_tags_without_resgistering`

* Log updated note instead of original one

* Revert merging imported tags

But keep old note tags if no new ones are provided.
2022-08-24 16:04:32 +10:00
Matthias Metelka
d1cbb86178
Default input setting in fields dialog (#1987)
* Introduce field setting to use plain text editor by default

* Remove leftover function from #1476

* Use boolean instead of string

* Simplify clear_other_field_duplicates

* Convert plain text key to camelCase

* Move HTML item below the existing checkbox, instead of to the right (dae)

Showing it on the right is more space efficient, but feels a bit
cluttered IMHO.
2022-08-18 12:30:18 +10:00
Damien Elmes
bd2f866c68 Fix {{CardFlags}} not working with flags 5-7
https://forums.ankiweb.net/t/please-activate-flags-5-7-on-the-cardflag/22143
2022-08-08 10:56:07 +10:00
Damien Elmes
f017525ea8 Handle deeply nested decks 2022-08-03 14:37:56 +10:00
Damien Elmes
340da23104 Wrap type answer output in code tag; remove extra div
This would make more sense as a class on the outer div, but existing
templates reference code#typeans, so avoid breaking things for now.
2022-07-22 20:37:56 +10:00
Damien Elmes
ad8c23fc3d Remove resolved comment 2022-07-22 20:06:45 +10:00
Damien Elmes
09eef8e746 Spelling fix 2022-07-22 19:32:31 +10:00
Damien Elmes
1e0be26b7e Partially migrate type answer to backend
Partially completes #1068, and will allow mobile clients to drop
their separate diff-match-patch imports. Does not yet try to handle
case folding or combining-char stripping, and leaves some of the outer
HTML wrapping up to the frontend for now.

The logic for rendering the provided string has changed: missing chars
are now only inserted if they follow a correct section, and the original
text is shown instead of hyphens. This is an experiment, and can be
changed if it's not well received.
2022-07-22 19:28:34 +10:00
RumovZ
173a5bfed5
Improve temporary table handling (#1976)
* Use all_cards_for_search() helper

* Add CardTableGuard

* Add for_each_card_in_search() helper

* Add all_cards_for_search_in_order() helper

* Add all_cards_for_ids() helper

* Return siblings for bury instead of only searching

* Remove redundant clear_searched_cards_table() calls

* Add with_searched_cards_table()

* Remove false comment

* Add NoteTableGuard

* Add with_ids_in_searched_notes_table() helper

* Make some last routines use table helpers
2022-07-22 17:51:26 +10:00
RumovZ
09841c7c0f
Populate media DB on import (#1977)
* Refactor MediaManager transactions

* Add media entries when importing

* Fix legacy apkg import not setting csum

Also test for this in the roundtrip test.

* Avoid reallocating MediaCopier's buffer

* Make sha1 optional (dae)
2022-07-22 17:50:15 +10:00
RumovZ
b9fd6688d2
Fix field check for cloze conditional (#1975) 2022-07-20 20:26:26 +10:00
RumovZ
5f9fa51026
Fix missing limits field in tests (#1973) 2022-07-19 18:51:32 +10:00
RumovZ
cc929687ae
Deck-specific Limits (#1955)
* Add deck-specific limits to DeckNormal

* Add deck-specific limits to schema11

* Add DeckLimitsDialog

* deck_limits_qt6.py needs to be a symlink

* Clear duplicate deck setting keys on downgrade

* Export deck limits when exporting with scheduling

* Revert "deck_limits_qt6.py needs to be a symlink"

This reverts commit 4ee7be1e10c4e8c49bb20de3bf45ac18b5e2d4f6.

* Revert "Add DeckLimitsDialog"

This reverts commit eb0e2a62d33df0b518d9204a27b09e97966ce82a.

* Add day limits to DeckNormal

* Add deck and day limits mock to deck options

* Revert "Add deck and day limits mock to deck options"

This reverts commit 0775814989e8cb486483d06727b1af266bb4513a.

* Add Tabs component for daily limits

* Add borders to tabs component

* Revert "Add borders to tabs component"

This reverts commit aaaf5538932540f944d92725c63bb04cfe97ea14.

* Implement tabbed limits properly

* Add comment to translations

* Update rslib/src/decks/limits.rs

Co-authored-by: Damien Elmes <dae@users.noreply.github.com>

* Fix camel case in clear_other_duplicates()

* day_limit → current_limit

* Also import day limits

* Remember last used day limits

* Add day limits to schema 11

* Tweak comment (dae)

* Exclude day limit in export (dae)

* Tweak tab wording (dae)

* Update preset limits on preset change

* Explain tabs in tooltip (dae)

* Omit deck and today limits if v2 is enabled

* Preserve deck limit when switching to today limit
2022-07-19 18:27:25 +10:00
Matthias Metelka
d1ba48bc48
Expose cloze text as HTML attribute on question side (#1968)
* Expose cloze text as attribute on front side

* Update test_models.py

* Update template_filters.rs

* Escape HTML for data-attribute

* Use minimal HTML encoding in Rust

to match Python's html.escape and pass tests.

* Rename attribute to data-cloze

to make it more generic.

* Run formatter

* Revert to using Rust encode_attribute and add helper function for tests
2022-07-19 04:22:57 +10:00
RumovZ
f6390e9455
Adjust remaining steps after config update (#1956)
* Adjust remaining steps after config update

* Handle relearning steps separately

Also refactor a lot.

* Also adjust remaining steps after deck change

* Test step adjustment after config update

* Fix `SearchBuilder::(re)learning_cards()`

* Fix step adjustment after deck change

* Test step adjustment after deck change

* Fix test name

* Readjust remaining steps according to last delay

Also atomize tests and add some tooling.
2022-07-14 11:24:34 +10:00
RumovZ
8c515e316e
Template err improvements (#1953)
* Throw error for unknown condition fields as well

So if 'foo' is not a field, refuse to save a template containing
`{{#foo}}bar{{/foo}}`. Previously, only `{{foo}}` would be checked.
As a side effect, templates which *only* contain fields as conditions
may be saved. Meh.

* Display template errors in q/a columns only

So the affected browser row remains active and the user can fix the
template more easily.

* Specify if error occured in a browser template

* Minor wording tweak (dae)

There's an argument for using the exact wording as well, but this just
reads a little more naturally to me.
2022-07-09 13:00:03 +10:00
Sam Penny
9dcceff4af
Remember previous choices in reposition dialog (#1950)
* remember previous choices in reposition dialog

* remember previous choice for randomize option as well

* fix failing test
2022-07-08 11:28:38 +10:00
RumovZ
2bbd1fca2a
Skip card generation if fronts remain unchanged (#1949)
Closes #1945
2022-07-06 19:26:14 +10:00
RumovZ
bebc6fd23a
Disregard manual reschedulings in introduced:x (#1932) 2022-06-27 17:20:36 +10:00
Damien Elmes
5ea78e1c8e Since DupeResolution is in CsvMetadata, we don't need to pass it separately
Follow-up to #1930
2022-06-27 17:15:54 +10:00
Damien Elmes
67e4edcd8b Expose backend_proto publicly for AnkiDroid, and rename to pb
We were aliasing it on import half the time anyway
2022-06-27 15:27:53 +10:00
Damien Elmes
6ff159304d Fix backups failing on Android 2022-06-27 15:27:51 +10:00
RumovZ
b6a7760cb4
Restore and save last dupe resolution setting (#1930)
* Restore dupe resolution setting

* Save dupe resolution setting

* Push config logic into backend (dae)
2022-06-24 15:10:06 +10:00
RumovZ
8478492190
Check ids when gathering data (#1928)
This will throw an error if a card, note or revlog id from the future
is found during apkg import or export.
2022-06-24 13:56:52 +10:00
RumovZ
1b8bab73d1
Csv import tweaks (#1920)
* Don't use special label for tags column

This was setting the label of whichever column the user chose for tags
to "Tags". Showing field content is more helpful.

* Map tags column automatically like fields
2022-06-22 09:44:12 +10:00
Damien Elmes
4d51ee8a64 Fix number of days in a (standard) year
https://forums.ankiweb.net/t/prop-ivl-faulty/20665
2022-06-13 09:09:27 +10:00
Damien Elmes
f15e294981 Fix collection load failure on Android
Loading a larger collection with V16 enabled was failing in openCollection
with error 6410, similar to https://github.com/simolus3/drift/issues/876
2022-06-10 23:21:35 +10:00
RumovZ
6da5e5b042
CSV import/export fixes and features (#1898)
* Fix footer moving upwards

* Fix column detection

Was broken because escaped line breaks were not considered.
Also removes delimiter detection on `#columns:` line. User must use tabs
or set delimiter beforehand.

* Add CSV preview

* Parse `#tags column:`

* Optionally export deck and notetype with CSV

* Avoid clones in CSV export

* Prevent bottom of page appearing under footer (dae)

* Increase padding to 1em (dae)

With 0.5em, when a vertical scrollbar is shown, it sits right next to
the right edge of the content, making it look like there's no right
margin.

* Experimental changes to make table fit+scroll (dae)

- limit individual cells to 15em, and show ellipses when truncated
- limit total table width to body width, so that inner table is shown
with scrollbar
- use class rather than id - ids are bad practice in Svelte components,
as more than one may be displayed on a single page

* Skip importing foreign notes with filtered decks

Were implicitly imported into the default deck before.
Also some refactoring to fetch deck ids and names beforehand.

* Hide spacer below hidden field mapping

* Fix guid being replaced when updating note

* Fix dupe identity check

Canonify tags before checking if dupe is identical, but only add update
tags later if appropriate.

* Fix deck export for notes with missing card 1

* Fix note lines starting with `#`

csv crate doesn't support escaping a leading comment char. :(

* Support import/export of guids

* Strip HTML from preview rows

* Fix initially set deck if current is filtered

* Make isHtml toggle reactive

* Fix `html_to_text_line()` stripping sound names

* Tweak export option labels

* Switch to patched rust-csv fork

Fixes writing lines starting with `#`, so revert 5ece10ad05f331.

* List column options with first column field

* Fix flag for exports with HTML stripped
2022-06-09 10:28:01 +10:00
RumovZ
42cbe42f06
Plaintext import/export (#1850)
* Add crate csv

* Add start of csv importing on backend

* Add Menomosyne serializer

* Add csv and json importing on backend

* Add plaintext importing on frontend

* Add csv metadata extraction on backend

* Add csv importing with GUI

* Fix missing dfa file in build

Added compile_data_attr, then re-ran cargo/update.py.

* Don't use doubly buffered reader in csv

* Escape HTML entities if CSV is not HTML

Also use name 'is_html' consistently.

* Use decimal number as foreign ease (like '2.5')

* ForeignCard.ivl → ForeignCard.interval

* Only allow fixed set of CSV delimiters

* Map timestamp of ForeignCard to native due time

* Don't trim CSV records

* Document use of empty strings for defaults

* Avoid creating CardGenContexts for every note

This requires CardGenContext to be generic, so it works both with an
owned and borrowed notetype.

* Show all accepted file types  in import file picker

* Add import_json_file()

* factor → ease_factor

* delimter_from_value → delimiter_from_value

* Map columns to fields, not the other way around

* Fallback to current config for csv metadata

* Add start of new import csv screen

* Temporary fix for compilation issue on Linux/Mac

* Disable jest bazel action for import-csv

Jest fails with an error code if no tests are available, but this would
not be noticable on Windows as Jest is not run there.

* Fix field mapping issue

* Revert "Temporary fix for compilation issue on Linux/Mac"

This reverts commit 21f8a261408cdae49ec031aa21a1b659c4f66d82.

* Add HtmlSwitch and move Switch to components

* Fix spacing and make selectors consistent

* Fix shortcut tooltip

* Place import button at the top with path

* Fix meta column indices

* Remove NotetypeForString

* Fix queue and type of foreign cards

* Support different dupe resolution strategies

* Allow dupe resolution selection when importing CSV

* Test import of unnormalized text

Close  #1863.

* Fix logging of foreign notes

* Implement CSV exports

* Use db_scalar() in notes_table_len()

* Rework CSV metadata

- Notetypes and decks are either defined by a global id or by a column.
- If a notetype id is provided, its field map must also be specified.
- If a notetype column is provided, fields are now mapped by index
instead of name at import time. So the first non-meta column is used for
the first field of every note, regardless of notetype. This makes
importing easier and should improve compatiblity with files without a
notetype column.
- Ensure first field can be mapped to a column.
- Meta columns must be defined as `#[meta name]:[column index]` instead
of in the `#columns` tag.
- Column labels contain the raw names defined by the file and must be
prettified by the frontend.

* Adjust frontend to new backend column mapping

* Add force flags for is_html and delimiter

* Detect if CSV is HTML by field content

* Update dupe resolution labels

* Simplify selectors

* Fix coalescence of oneofs in TS

* Disable meta columns from selection

Plus a lot of refactoring.

* Make import button stick to the bottom

* Write delimiter and html flag into csv

* Refetch field map after notetype change

* Fix log labels for csv import

* Log notes whose deck/notetype was missing

* Fix hiding of empty log queues

* Implement adding tags to all notes of a csv

* Fix dupe resolution not being set in log

* Implement adding tags to updated notes of a csv

* Check first note field is not empty

* Temporary fix for build on Linux/Mac

* Fix inverted html check (dae)

* Remove unused ftl string

* Delimiter → Separator

* Remove commented-out line

* Don't accept .json files

* Tweak tag ftl strings

* Remove redundant blur call

* Strip sound and add spaces in csv export

* Export HTML by default

* Fix unset deck in Mnemosyne import

Also accept both numbers and strings for notetypes and decks in JSON.

* Make DupeResolution::Update the default

* Fix missing dot in extension

* Make column indices 1-based

* Remove StickContainer from TagEditor

Fixes line breaking, border and z index on ImportCsvPage.

* Assign different key combos to tag editors

* Log all updated duplicates

Add a log field for the true number of found notes.

* Show identical notes as skipped

* Split tag-editor into separate ts module (dae)

* Add progress for CSV export

* Add progress for text import

* Tidy-ups after tag-editor split (dae)

- import-csv no longer depends on editor
- remove some commented lines
2022-06-01 20:26:16 +10:00
dobefore
b4cffe2339
rust impl stdError trait (#1890)
* impl stdError trait

* Update CONTRIBUTORS
2022-05-31 17:52:12 +10:00
Damien Elmes
5d835ae813 Always generate missing cards when saving card templates
https://forums.ankiweb.net/t/anki-doesnt-generate-cards-after-un-doing-selective-card-generation/20283
2022-05-26 12:44:57 +10:00
Damien Elmes
82196753ec Rework display of available cards in custom study
In v3, it's more informative to show the count of child decks separately,
since increasing the limit of the current deck does not increase the limits
of child decks. When we rework the decks list in the future, a tooltip
will hopefully provide an easier way for users to see where cards are
available, and where limits are being applied.

Closes #1868
2022-05-20 17:52:02 +10:00
RumovZ
6c807c1b2e
Fix review queue if limit is reached (#1855) 2022-05-10 12:11:35 +10:00
Damien Elmes
d946e5ddd5
Fix for crash with invalid dates on Windows (#1837)
https://forums.ankiweb.net/t/bug-report-crashing-when-opening-deck-browser/19768

Caused by a note mtime that was 1000x larger than it should have been.
Check DB will now fix this case (but there are others it still does not
cover, such as invalid card/note IDs).

https://docs.rs/chrono/0.4.19/x86_64-pc-windows-msvc/src/chrono/sys/windows.rs.html#128
2022-05-07 10:30:23 +10:00
Damien Elmes
c8a4e5ea22 Add more progress + abort points to export code
The old `media_files_did_export` hook has been kept around for use with
the legacy apkg exporter (an add-on uses it), and a new 
`legacy_export_progress` hook has been added so we can get progress
from the new colpkg exporter until we move over fully to the new code.
2022-05-06 15:32:23 +10:00
RumovZ
99c4ec93d2
Fix some apkg import issues (#1836)
* Allow gathering of orphaned decks

* Skip redundant media checking in apkg import
2022-05-06 09:22:15 +10:00
Damien Elmes
ac1cbaf719 Fix implicit transaction not being cleared on transact() failure
"release foo" clears the implicit outer transaction, but "rollback to foo"
does not.
2022-05-05 13:39:41 +10:00
Damien Elmes
1297beff63 Minor progress tweaks to apkg importing
- Set progress as soon as possible; previously the extracting and
gathering steps happened prior to the first progress.
- Add "extracting" and "gathering" progress steps, so that large imports
show more feedback in the early stage, and can be more quickly
interrupted.
2022-05-05 12:23:05 +10:00
RumovZ
5f9451f547
Add apkg import/export on backend (#1743)
* Add apkg export on backend

* Filter out missing media-paths at write time

* Make TagMatcher::new() infallible

* Gather export data instead of copying directly

* Revert changes to rslib/src/tags/

* Reuse filename_is_safe/check_filename_safe()

* Accept func to produce MediaIter in export_apkg()

* Only store file folder once in MediaIter

* Use temporary tables for gathering

export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.

* Use schedule_as_new() to reset cards

* ExportData → ExchangeData

* Ignore ascii case when filtering system tags

* search_notes_cards_into_table →

search_cards_of_notes_into_table

* Start on apkg importing on backend

* Fix due dates in days for apkg export

* Refactor import-export/package

- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.

* Add SafeMediaEntry for deserialized MediaEntries

* Prepare media based on checksums

- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.

* Handle encoding in `replace_media_refs()`

* Add trait to keep down cow boilerplate

* Add notetypes immediately instaed of preparing

* Move target_col into Context

* Add notes immediately instaed of preparing

* Note id, not guid of conflicting notes

* Add import_decks()

* decks_configs → deck_configs

* Add import_deck_configs()

* Add import_cards(), import_revlog()

* Use dyn instead of generic for media_fn

Otherwise, would have to pass None with type annotation in the default
case.

* Fix signature of import_apkg()

* Fix search_cards_of_notes_into_table()

* Test new functions in text.rs

* Add roundtrip test for apkg (stub)

* Keep source id of imported cards (or skip)

* Keep source ids of imported revlog (or skip)

* Try to keep source ids of imported notes

* Make adding notetype with id undoable

* Wrap apkg import in transaction

* Keep source ids of imported deck configs (or skip)

* Handle card due dates and original due/did

* Fix importing cards/revlog

Card ids are manually uniquified.

* Factor out card importing

* Refactor card and revlog importing

* Factor out card importing

Also handle missing parents .

* Factor out note importing

* Factor out media importing

* Maybe upgrade scheduler of apkg

* Fix parent deck gathering

* Unconditionally import static media

* Fix deck importing edge cases

Test those edge cases, and add some global test helpers.

* Test note importing

* Let import_apkg() take a progress func

* Expand roundtrip apkg test

* Use fat pointer to avoid propogating generics

* Fix progress_fn type

* Expose apkg export/import on backend

* Return note log when importing apkg

* Fix archived collection name on apkg import

* Add CollectionOpWithBackendProgress

* Fix wrong Interrupted Exception being checked

* Add ClosedCollectionOp

* Add note ids to log and strip HTML

* Update progress when checking incoming media too

* Conditionally enable new importing in GUI

* Fix all_checksums() for media import

Entries of deleted files are nulled, not removed.

* Make apkg exporting on backend abortable

* Return number of notes imported from apkg

* Fix exception printing for QueryOp as well

* Add QueryOpWithBackendProgress

Also support backend exporting progress.

* Expose new apkg and colpkg exporting

* Open transaction in insert_data()

Was slowing down exporting by several orders of magnitude.

* Handle zstd-compressed apkg

* Add legacy arg to ExportAnkiPackage

Currently not exposed on the frontend

* Remove unused import in proto file

* Add symlink for typechecking of import_export_pb2

* Avoid kwargs in pb message creation, so typechecking is not lost

Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign 
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.

* Avoid re-exporting protobuf msgs we only use internally

* Stop after one test failure

mypy often fails much faster than pylint

* Avoid an extra allocation when extracting media checksums

* Update progress after prepare_media() finishes

Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.

* Show progress of note imports

Note import is the slowest part, so showing progress here makes the UI
feel more responsive.

* Reset filtered decks at import time

Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.

We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).

https://github.com/ankitects/anki/pull/1743/files#r839346423

* Fix a corner-case where due dates were shifted by a day

This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.

* Log conflicting note in remapped nt case

* take_fields() → into_fields()

* Alias `[u8; 20]` with `Sha1Hash`

* Truncate logged fields

* Rework apkg note import tests

- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)

* Fix sorting of imported decks

Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.

* target[_id]s → existing_card[_id]s

* export_collection_extracting_media() → ...

export_into_collection_file()

* target_already_exists→card_ordinal_already_exists

* Add search_cards_of_notes_into_table.sql

* Imrove type of apkg export selector/limit

* Remove redundant call to mod_schema()

* Parent tooltips to mw

* Fix a crash when truncating note text

String::truncate() is a bit of a footgun, and I've hit this before
too :-)

* Remove ExportLimit in favour of separate classes

* Remove OpWithBackendProgress and ClosedCollectionOp

Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.

Also fix aborting of colpkg exports, which slipped through in #1817.

* Tidy up import log

* Avoid QDialog.exec()

* Default to excluding scheuling for deck list deck

* Use IncrementalProgress in whole import_export code

* Compare checksums when importing colpkgs

* Avoid registering changes if hashes are not needed

* ImportProgress::Collection → ImportProgress::File

* Make downgrading apkgs depend on meta version

* Generalise IncrementableProgress

And use it in entire import_export code instead.

* Fix type complexity lint

* Take count_map for IncrementableProgress::get_inner

* Replace import/export env with Shift click

* Accept all args from update() for backend progress

* Pass fields of ProgressUpdate explicitly

* Move update_interval into IncrementableProgress

* Outsource incrementing into Incrementor

* Mutate ProgressUpdate in progress_update callback

* Switch import/export legacy toggle to profile setting

Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.

One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.

* Show extension in export dialog

* Continue to provide separate options for schema 11+18 colpkg export

* Default to colpkg export when using File>Export

* Improve appearance of combo boxes when switching between apkg/colpkg

+ Deal with long deck names

* Convert newlines to spaces when showing fields from import

Ensures each imported note appears on a separate line

* Don't separate total note count from the other summary lines

This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.

* Fix 'deck not normal' error when importing a filtered deck for the 2nd time

* Fix [Identical] being shown on first import

* Revert "Continue to provide separate options for schema 11+18 colpkg export"

This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.

Will use a different approach

* Move legacy support into a separate exporter option; add to apkg export

* Adjust 'too new' message to also apply to .apkg import case

* Show a better message when attempting to import new apkg into old code

Previously the user could end seeing a message like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

Unfortunately we can't retroactively fix this for older clients.

* Hide legacy support option in older exporting screen

* Reflect change from paths to fnames in type & name

* Make imported decks normal at once

Then skip special casing in update_deck(). Also skip updating
description if new one is empty.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-05-02 21:12:46 +10:00
Damien Elmes
d2ec004ac3 Fix custom study applying previously-selected limit
extend_limits() mutates deck, so we need to re-fetch it.

Closes #1829
2022-05-02 20:57:29 +10:00
Abdo
a3d9a25d18
Extends the SearchNode interface to support field and unqualified searches (#1821)
* Add field search option to SearchNode

* Add unqualified/literal search option to SearchNode

* Rename text to literal_text

* Interpret field name parameter literally
2022-04-28 14:09:51 +10:00