Commit Graph

9464 Commits

Author SHA1 Message Date
Damien Elmes
ac1cbaf719 Fix implicit transaction not being cleared on transact() failure
"release foo" clears the implicit outer transaction, but "rollback to foo"
does not.
2022-05-05 13:39:41 +10:00
Damien Elmes
1297beff63 Minor progress tweaks to apkg importing
- Set progress as soon as possible; previously the extracting and
gathering steps happened prior to the first progress.
- Add "extracting" and "gathering" progress steps, so that large imports
show more feedback in the early stage, and can be more quickly
interrupted.
2022-05-05 12:23:05 +10:00
Damien Elmes
bc6ede7c11 Fix & showing up in MathJax
https://forums.ankiweb.net/t/anki-2-1-52-release-candidate/19412/22

Accidentally introduced in #1818
2022-05-03 14:21:31 +10:00
Damien Elmes
fdbe211539 Support new import code in drag/drop + file double-click cases
https://forums.ankiweb.net/t/anki-2-1-52-release-candidate/19412/24
2022-05-03 14:09:40 +10:00
Matthias Metelka
567b05b731
Fix scrollbars showing on bottom bar (#1833) 2022-05-03 13:57:33 +10:00
Damien Elmes
63f05b2b1e Update translations 2022-05-02 21:14:10 +10:00
RumovZ
5f9451f547
Add apkg import/export on backend (#1743)
* Add apkg export on backend

* Filter out missing media-paths at write time

* Make TagMatcher::new() infallible

* Gather export data instead of copying directly

* Revert changes to rslib/src/tags/

* Reuse filename_is_safe/check_filename_safe()

* Accept func to produce MediaIter in export_apkg()

* Only store file folder once in MediaIter

* Use temporary tables for gathering

export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.

* Use schedule_as_new() to reset cards

* ExportData → ExchangeData

* Ignore ascii case when filtering system tags

* search_notes_cards_into_table →

search_cards_of_notes_into_table

* Start on apkg importing on backend

* Fix due dates in days for apkg export

* Refactor import-export/package

- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.

* Add SafeMediaEntry for deserialized MediaEntries

* Prepare media based on checksums

- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.

* Handle encoding in `replace_media_refs()`

* Add trait to keep down cow boilerplate

* Add notetypes immediately instaed of preparing

* Move target_col into Context

* Add notes immediately instaed of preparing

* Note id, not guid of conflicting notes

* Add import_decks()

* decks_configs → deck_configs

* Add import_deck_configs()

* Add import_cards(), import_revlog()

* Use dyn instead of generic for media_fn

Otherwise, would have to pass None with type annotation in the default
case.

* Fix signature of import_apkg()

* Fix search_cards_of_notes_into_table()

* Test new functions in text.rs

* Add roundtrip test for apkg (stub)

* Keep source id of imported cards (or skip)

* Keep source ids of imported revlog (or skip)

* Try to keep source ids of imported notes

* Make adding notetype with id undoable

* Wrap apkg import in transaction

* Keep source ids of imported deck configs (or skip)

* Handle card due dates and original due/did

* Fix importing cards/revlog

Card ids are manually uniquified.

* Factor out card importing

* Refactor card and revlog importing

* Factor out card importing

Also handle missing parents .

* Factor out note importing

* Factor out media importing

* Maybe upgrade scheduler of apkg

* Fix parent deck gathering

* Unconditionally import static media

* Fix deck importing edge cases

Test those edge cases, and add some global test helpers.

* Test note importing

* Let import_apkg() take a progress func

* Expand roundtrip apkg test

* Use fat pointer to avoid propogating generics

* Fix progress_fn type

* Expose apkg export/import on backend

* Return note log when importing apkg

* Fix archived collection name on apkg import

* Add CollectionOpWithBackendProgress

* Fix wrong Interrupted Exception being checked

* Add ClosedCollectionOp

* Add note ids to log and strip HTML

* Update progress when checking incoming media too

* Conditionally enable new importing in GUI

* Fix all_checksums() for media import

Entries of deleted files are nulled, not removed.

* Make apkg exporting on backend abortable

* Return number of notes imported from apkg

* Fix exception printing for QueryOp as well

* Add QueryOpWithBackendProgress

Also support backend exporting progress.

* Expose new apkg and colpkg exporting

* Open transaction in insert_data()

Was slowing down exporting by several orders of magnitude.

* Handle zstd-compressed apkg

* Add legacy arg to ExportAnkiPackage

Currently not exposed on the frontend

* Remove unused import in proto file

* Add symlink for typechecking of import_export_pb2

* Avoid kwargs in pb message creation, so typechecking is not lost

Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign 
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.

* Avoid re-exporting protobuf msgs we only use internally

* Stop after one test failure

mypy often fails much faster than pylint

* Avoid an extra allocation when extracting media checksums

* Update progress after prepare_media() finishes

Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.

* Show progress of note imports

Note import is the slowest part, so showing progress here makes the UI
feel more responsive.

* Reset filtered decks at import time

Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.

We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).

https://github.com/ankitects/anki/pull/1743/files#r839346423

* Fix a corner-case where due dates were shifted by a day

This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.

* Log conflicting note in remapped nt case

* take_fields() → into_fields()

* Alias `[u8; 20]` with `Sha1Hash`

* Truncate logged fields

* Rework apkg note import tests

- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)

* Fix sorting of imported decks

Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.

* target[_id]s → existing_card[_id]s

* export_collection_extracting_media() → ...

export_into_collection_file()

* target_already_exists→card_ordinal_already_exists

* Add search_cards_of_notes_into_table.sql

* Imrove type of apkg export selector/limit

* Remove redundant call to mod_schema()

* Parent tooltips to mw

* Fix a crash when truncating note text

String::truncate() is a bit of a footgun, and I've hit this before
too :-)

* Remove ExportLimit in favour of separate classes

* Remove OpWithBackendProgress and ClosedCollectionOp

Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.

Also fix aborting of colpkg exports, which slipped through in #1817.

* Tidy up import log

* Avoid QDialog.exec()

* Default to excluding scheuling for deck list deck

* Use IncrementalProgress in whole import_export code

* Compare checksums when importing colpkgs

* Avoid registering changes if hashes are not needed

* ImportProgress::Collection → ImportProgress::File

* Make downgrading apkgs depend on meta version

* Generalise IncrementableProgress

And use it in entire import_export code instead.

* Fix type complexity lint

* Take count_map for IncrementableProgress::get_inner

* Replace import/export env with Shift click

* Accept all args from update() for backend progress

* Pass fields of ProgressUpdate explicitly

* Move update_interval into IncrementableProgress

* Outsource incrementing into Incrementor

* Mutate ProgressUpdate in progress_update callback

* Switch import/export legacy toggle to profile setting

Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.

One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.

* Show extension in export dialog

* Continue to provide separate options for schema 11+18 colpkg export

* Default to colpkg export when using File>Export

* Improve appearance of combo boxes when switching between apkg/colpkg

+ Deal with long deck names

* Convert newlines to spaces when showing fields from import

Ensures each imported note appears on a separate line

* Don't separate total note count from the other summary lines

This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.

* Fix 'deck not normal' error when importing a filtered deck for the 2nd time

* Fix [Identical] being shown on first import

* Revert "Continue to provide separate options for schema 11+18 colpkg export"

This reverts commit 8f0b2c175f4794d642823b60414d142a12768441.

Will use a different approach

* Move legacy support into a separate exporter option; add to apkg export

* Adjust 'too new' message to also apply to .apkg import case

* Show a better message when attempting to import new apkg into old code

Previously the user could end seeing a message like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

Unfortunately we can't retroactively fix this for older clients.

* Hide legacy support option in older exporting screen

* Reflect change from paths to fnames in type & name

* Make imported decks normal at once

Then skip special casing in update_deck(). Also skip updating
description if new one is empty.

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-05-02 21:12:46 +10:00
Damien Elmes
d2ec004ac3 Fix custom study applying previously-selected limit
extend_limits() mutates deck, so we need to re-fetch it.

Closes #1829
2022-05-02 20:57:29 +10:00
RumovZ
1e02980133
Fix previewer runtime error "deleted" (#1830) 2022-05-02 10:40:44 +10:00
Abdo
a3d9a25d18
Extends the SearchNode interface to support field and unqualified searches (#1821)
* Add field search option to SearchNode

* Add unqualified/literal search option to SearchNode

* Rename text to literal_text

* Interpret field name parameter literally
2022-04-28 14:09:51 +10:00
Damien Elmes
27f2e39ff9
Drop default Show Answer/Good highlight (#1820)
Feedback welcome. Discussion at
https://forums.ankiweb.net/t/2-1-5-border-around-good/19175

Partially reverts 301c9587d1
2022-04-27 19:26:16 +10:00
Damien Elmes
6f0d86b9a9 Cap tag matches to 10 on macOS/Qt5 to work around Chromium perf issue
Closes #1825
2022-04-27 19:23:07 +10:00
Damien Elmes
a989e50838 Update translations 2022-04-25 14:35:35 +10:00
Damien Elmes
f60de39015 Fix due graph showing wrong date for review cards with resched off
Since we are using the original due date instead of the current one,
the learning check needs to be based on the card type, not its current
queue.

https://forums.ankiweb.net/t/anki-2-1-51-release-candidate/18942/22
2022-04-25 14:14:08 +10:00
Henrik Giesel
b8eabfd622
Remove onInput call in input handler (#1819)
This was implemented by me to prevent seemingly empty fields,
which however still contain some HTML (e.g. <div></div>) and
thus will trigger in the templates.
However it only worked in very limited cases, maybe we could
solve this better by indicating whether a field has content somehow.
2022-04-25 13:56:05 +10:00
Henrik Giesel
2be1f4c56d
Instead of transforming to {\lt}, transform to &lt; (#1818)
* Instead of transforming to {\lt}, transform to &lt;

- In Mathjax editor
- This way you can also use Mathjax convenience shortcuts like <=> in
  chemistry mode: \ce{<=>}

* Remove unused translateEntitiesToMathjax() (dae)

https://github.com/ankitects/anki/pull/1818#discussion_r857238310
2022-04-25 13:42:54 +10:00
Damien Elmes
ee70006ec4 Reset zoom on page transition
https://forums.ankiweb.net/t/2-1-50-zooming-in-while-reviewing-results-in-an-unusably-huge-deck-list/19283
2022-04-22 15:45:50 +10:00
RumovZ
aea7eb667e
Fix progress of chained operations (#1817)
* Fix progress of chained operations

Especially aborting colpkg import.

* Notify about missing progress dialog
2022-04-22 14:47:17 +10:00
Matthias Metelka
2b739d76b2
Align list items to start (#1816) 2022-04-22 14:18:11 +10:00
Matthias Metelka
770fabd51b
Remove unnecessary night mode rule for typeans (#1815)
which caused https://forums.ankiweb.net/t/shaddow-below-cloze-type-area-in-night-mode/19287/2
2022-04-22 14:17:43 +10:00
Damien Elmes
280c06425d Increase compression on zstd packages 2022-04-22 13:52:24 +10:00
Damien Elmes
e6d990a44e Fix cwd changing on export
No longer required after #1656
2022-04-21 16:50:05 +10:00
Abdo
6ea75df292
Fix pasting of local links to unsupported types that contain spaces (#1813)
This change causes local links to unsupported formats like PDF to be
handled by _processUrls() instead of _processText, converting them to
correct hyperlinks in case they contain spaces.
2022-04-21 14:10:53 +10:00
Damien Elmes
6f764aeda4 Fix custom browser font not being honored
https://forums.ankiweb.net/t/on-anki-2-1-50-browser-table-font-override-is-not-respected/19259
2022-04-21 13:28:22 +10:00
Damien Elmes
ef51bcefd6 Bump bazel version
Closes #1808
2022-04-20 21:53:59 +10:00
Damien Elmes
031fc625fd Update Mac build to Qt 6.3 2022-04-20 21:33:31 +10:00
Damien Elmes
30d55aa553 Only require ANKI_WAYLAND if frozen
Accidentally changed `and` to `or` while testing 7115caab13.
2022-04-20 19:41:41 +10:00
Damien Elmes
0b03919f45 Upgrade to Qt 6.3
Fixes #1753
Fixes #1770
2022-04-20 19:12:10 +10:00
Damien Elmes
ff9aed91de Update translations 2022-04-20 19:12:10 +10:00
Damien Elmes
da0cbfec32 Add workaround for unwanted CF_HTML header when pasting on Windows
Fixes #1751 for now; hopefully we can roll this change back once 6.3.1
is out in 3-4 weeks.
2022-04-20 17:23:11 +10:00
Damien Elmes
3031219a34 Add workaround to ensure internal pastes flagged on Windows/Qt6
Fixes #1793
2022-04-20 17:05:24 +10:00
Damien Elmes
dbcbff1fb1 Bump version 2022-04-20 16:40:49 +10:00
Damien Elmes
7115caab13 Default to X11 on Qt6 builds as well
Qt6 no longer appears to require QT_QPA_PLATFORM to be set.

https://forums.ankiweb.net/t/blury-interface-with-gnome-wayland-and-fractional-scaling/19196/4
2022-04-20 14:41:01 +10:00
Henrik Giesel
cf78a555c6
Add some component comment (#1779)
* Add a component comment for NoteEditor

* Move comments above component template

* EditorField -> EditingArea
2022-04-19 17:20:26 +10:00
Damien Elmes
adbe96ca00
Load page script after body created (#1802)
document.body must exist by the time the top level Svelte component
initializes.

https://forums.ankiweb.net/t/2-1-50-editor-wont-show-when-addons-load-many-js-files/19036
2022-04-19 17:16:56 +10:00
Damien Elmes
a7cb5e210e Close browser before full sync or colpkg import/export
Closes #1797
2022-04-19 17:10:34 +10:00
Damien Elmes
856d4763a9 Switch strong/em back to b/i
In the absence of strong arguments either way, and given the fact that
strong/em has broken a few (perhaps rare) use-cases, reverting seems
prudent. But happy to revisit this again in the future if someone has
a convincing argument for why we should be doing it differently.

Closes #1807
2022-04-19 16:28:44 +10:00
Henrik Giesel
6017214042
Fix resetting of formatting modes when pressing modifer or dead keys (#1811)
* Fix shift removing bold formatting

* Remove input handler on selection change

We listened to "non-input events" mostly to catch events that
change the selection (like Ctrl-a, or arrow keys) anyway
2022-04-19 16:08:32 +10:00
Luka Warren
0835129a00
Fix issue #1114 by scrolling the user to the top of the page (#1810)
* Scroll stats to top when exporting (#1114)

It's obviously a bit of a "hacky" solution, since it's slightly jarring for users to scroll down, click export, then find themselves all the way at the top again, but it's somewhat less confusing than wondering why your PDF is broken :-)

* Clarified comment in stats.py (#1114)

* Apply scrolling fix to new stats screen, not legacy stats (dae)

Also wait for JS callback before saving the PDF, as JS is executed
asynchronously.
2022-04-19 15:39:32 +10:00
Damien Elmes
72c7d64876 Fix compatibility with older macOS versions
https://forums.ankiweb.net/t/anki-2-1-50-qt5-wont-open/19091
2022-04-15 17:35:05 +10:00
Damien Elmes
a9769813ba Add back support for custom mountpoint in card stats
The move to separate .html files broke our legacy card stats routine.

Related: d1d71ffdbb
2022-04-15 15:30:05 +10:00
Damien Elmes
24ab8ac423 Fix copying text to external apps when running on Gnome
Mutter seems to get confused if we mutate the keyboard in dataChanged().

Closes #1799
2022-04-15 12:09:56 +10:00
RumovZ
8ccadd96e6
Store last position when scheduling as review (#1800) 2022-04-14 18:53:58 +10:00
RumovZ
e29a5f779b
Fix selector index after programatic search (#1796) 2022-04-14 18:46:40 +10:00
Damien Elmes
29c48a60e0 Fix illegible calendar buttons in dark mode on Mac/Linux
https://forums.ankiweb.net/t/macos-apple-silicon-2-1-50-dark-mode-statistics-heatmap-buttons-dont-have-icons/18951

This button code is in need of a cleanup.
2022-04-13 13:53:45 +10:00
Damien Elmes
51a0641bba Sort field change without field change should mark schema modified
Closes #1789
2022-04-10 08:52:55 +10:00
Damien Elmes
0b2a226dc8 Add mock stdout/stderr for console=False case on Windows
PyInstaller provided a NulWriter when running in GUI mode, but stdout/
stderr now default to None with PyOxidizer.

Fixes #1786
2022-04-09 16:48:32 +10:00
Abdo
a99253b990
Allow passing args to anki-console.bat (#1785) 2022-04-09 14:40:26 +10:00
roxgib
77d14ddf1e
Add minimumContentsLength property to templatesBox (#1781)
Addresses #1777 by adding the `minimumContentsLength` property to the comboBox, which causes the size policy to not require expanding beyond that amount while still expanding if the user stretches the window manually.

Note that users who were previously experiencing large window widths due to the previous behaviour might see the same size again due the geometry being restored.
2022-04-09 13:55:03 +10:00
Aristotelis
8100e81789
Fix a number of bugs with add-on conflict resolution (#1780)
* Always enable manually installed add-ons

Ensures that manually installed add-ons are enabled after the installation, even if previously disabled.

Prevents scenarios where users could end up with no active add-on build (e.g. when switching between stable add-on builds distributed via AnkiWeb and betas distributed via GitHub).

* Improve type annotations

* Also enable disabled AnkiWeb add-ons upon interactive installation

Applies to add-ons that users actively install via their AnkiWeb ID. Updates are exempt, preserving whatever status add-ons were in.

* Prevent disabled add-ons from triggering conflicts

* Fix download_addons() not passing on force_enable argument (dae)
2022-04-09 13:51:59 +10:00