Commit Graph

1669 Commits

Author SHA1 Message Date
RumovZ
5781e86995
Don't propogate limit extensions in v3 scheduler (#1724) 2022-03-18 19:16:31 +10:00
Damien Elmes
5c1abb7f09 Add missing file header 2022-03-17 21:09:20 +10:00
Damien Elmes
dadb69a14b Fix some clippy lints + imports 2022-03-17 21:03:39 +10:00
Damien Elmes
bf8e70c70f Ensure partial colpkg file removed if export fails 2022-03-17 20:58:36 +10:00
Damien Elmes
f8ed4d89ba Add new error variant to frontend; ensure errors are mapped appropriately 2022-03-17 20:58:35 +10:00
Damien Elmes
30e53c51ab Tiny cleanup missing from previous commit
Had intended to apply it prior to pushing.
2022-03-17 17:49:18 +10:00
Damien Elmes
4687620f5e Expand normalization checks on import/export
The old Python code was only checking for NFC encoding, but we should
check for other issues like special filenames on windows (eg con.mp3)

- On export, the user is told to use Check Media if their media has
invalid filenames.
- On import, legacy packages will be transparently normalized. Since we're
doing the checks on export as well, any invalid names in a v3 package
are an error.
2022-03-17 17:31:19 +10:00
Damien Elmes
57a4495d92 Check for attempted path traversal on import 2022-03-17 16:40:52 +10:00
Damien Elmes
c2e8d89fc6
Colpkg fixes (#1722)
* Fix legacy colpkg import; disable v3 import/export; add roundtrip test

The test has revealed we weren't decompressing the media files on v3
import. That's easy to fix, but means all files need decompressing
even when they already exist, which is not ideal - it would be better
to store size/checksum in the metadata instead.

* Switch media and meta to protobuf; re-enable v3 import/export

- Fixed media not being decompressed on import
- The uncompressed size and checksum is now included for each media
entry, so that we can quickly check if a given file needs to be extracted.
We're still just doing a naive size comparison on colpkg import at the
moment, but we may want to use a checksum in the future, and will need
a checksum for apkg imports.
- Checksums can't be efficiently encoded in JSON, so the media list
has been switched to protobuf to reduce the the space requirements.
- The meta file has been switched to protobuf as well, for consistency.
This will mean any colpkg files exported with beta7 will be
unreadable.

* Avoid integer version comparisons

* Re-enable v3 test

* Apply suggestions from code review

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>

* Add export_colpkg() method to Collection

More discoverable, and easier to call from unit tests

* Split import/export code out into separate folders

Currently colpkg/*.rs contain some routines that will be useful for
apkg import/export as well; in the future we can refactor them into a
separate file in the parent module.

* Return a proper error when media import fails

This tripped me up when writing the earlier unit test - I had called
the equivalent of import_colpkg()?, and it was returning a string error
that I didn't notice. In practice this should result in the same text
being shown in the UI, but just skips the tooltip.

* Automatically create media folder on import

* Move roundtrip test into separate file; check collection too

* Remove zstd version suffix

Prevents a warning shown each time Rust Analyzer is used to check the
code.

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
2022-03-17 15:11:23 +10:00
Damien Elmes
ed9476c856 Update Rust deps
Primarily for https://blog.rust-lang.org/2022/03/08/cve-2022-24713.html
2022-03-15 16:51:52 +10:00
RumovZ
e759885734
Backend colpkg exporting (#1719)
* Implement colpkg exporting on backend

* Use exporting logic in backup.rs

* Refactor exporting.rs

* Add backend function to export collection

* Refactor backend/collection.rs

* Use backend for colpkg exporting

* Don't use default zip compression for media

* Add exporting progress

* Refactor media file writing

* Write dummy collections

* Localize dummy collection note

* Minimize dummy db size

* Use `NamedTempFile::new()` instead of `new_in`

* Drop redundant v2 dummy collection

* COLLECTION_VERSION -> PACKAGE_VERSION

* Split `lock_collection()` into two to drop flag

* Expose new colpkg in GUI

* Improve dummy collection message

* Please type checker

* importing-colpkg-too-new -> exporting-...

* Compress the media map in the v3 package (dae)

On collections with lots of media, it can grow into megabytes.

Also return an error in extract_media_file_names(), instead of masking
it as an optional.

* Store media map as a vector in the v3 package (dae)

This compresses better (eg 280kb original, 100kb hashmap, 42kb vec)

In the colpkg import case we don't need random access. When importing
an apkg, we will need to be able to fetch file data for a given media
filename, but the existing map doesn't help us there, as we need
filename->index, not index->filename.

* Ensure folders in the media dir don't break the file mapping (dae)
2022-03-15 16:48:02 +10:00
Damien Elmes
2d6dd0630f Switch card.original_position to a proto3 optional 2022-03-10 20:48:26 +10:00
Damien Elmes
f3e81c8a95 Move custom study tag and limit gathering+saving into the backend
Ideally this would have been in beta 6 :-) No add-ons appear to be
using customstudy.py/taglimit.py though, so it should hopefully not be
disruptive.

In the earlier custom study changes, we didn't get around to addressing
issue #1136. Now instead of trying to determine the maximum increase
to allow (which doesn't work correctly with nested decks), we just
present the total available to the user again, and let them decide. There's
plenty of room for improvement here still, but further work here might
be better done once we look into decoupling deck limits from deck presets.

Tags and available cards are fetched prior to showing the dialog now,
and will show a progress dialog if things take a while.

Tags are stored in an aux var now, so they don't inflate the deck
object size.
2022-03-10 16:23:03 +10:00
RumovZ
b9c3b12f71
Optionally restore original position and reset counts when forgetting (#1714)
* Add forget prompt with options

- Restore original position
- Reset reps and lapses

* Restore position when resetting for export

* Add config context to avoid passing keys

* Add routine to fetch defaults; use method-specific enum (dae)

* Keep original position by default (dae)

* Fix code completion for forget dialog (dae)

Needs to be a symbolic link to the generated file
2022-03-09 16:51:41 +10:00
RumovZ
c21e6e2b97
Disable full screen on Windows with OpenGL (#1715) 2022-03-09 14:21:54 +10:00
Damien Elmes
e6028cd9a4 Preserve old translation
7f076ee92a..9f37a0245c (r820447434)
2022-03-07 18:14:12 +10:00
RumovZ
f3c8857421
Backups (#1685)
* Add zstd dep

* Implement backend backup with zstd

* Implement backup thinning

* Write backup meta

* Use new file ending anki21b

* Asynchronously backup on collection close in Rust

* Revert "Add zstd dep"

This reverts commit 3fcb2141d2be15f907269d13275c41971431385c.

* Add zstd again

* Take backup col path from col struct

* Fix formatting

* Implement backup restoring on backend

* Normalize restored media file names

* Refactor `extract_legacy_data()`

A bit cumbersome due to borrowing rules.

* Refactor

* Make thinning calendar-based and gradual

* Consider last kept backups of previous stages

* Import full apkgs and colpkgs with backend

* Expose new backup settings

* Test `BackupThinner` and make it deterministic

* Mark backup_path when closing optional

* Delete leaky timer

* Add progress updates for restoring media

* Write restored collection to tempfile first

* Do collection compression in the background thread

This has us currently storing an uncompressed and compressed copy of
the collection in memory (not ideal), but means the collection can be
closed without waiting for compression to complete. On a large collection,
this takes a close and reopen from about 0.55s to about 0.07s. The old
backup code for comparison: about 0.35s for compression off, about
8.5s for zip compression.

* Use multithreading in zstd compression

On my system, this reduces the compression time of a large collection
from about 0.55s to 0.08s.

* Stream compressed collection data into zip file

* Tweak backup explanation

+ Fix incorrect tab order for ignore accents option

* Decouple restoring backup and full import

In the first case, no profile is opened, unless the new collection
succeeds to load.
In the second case, either the old collection is reloaded or the new one
is loaded.

* Fix number gap in Progress message

* Don't revert backup when media fails but report it

* Tweak error flow

* Remove native BackupLimits enum

* Fix type annotation

* Add thinning test for whole year

* Satisfy linter

* Await async backup to finish

* Move restart disclaimer out of backup tab

Should be visible regardless of the current tab.

* Write restored collection in chunks

* Refactor

* Write media in chunks and refactor

* Log error if removing file fails

* join_backup_task -> await_backup_completion

* Refactor backup.rs

* Refactor backup meta and collection extraction

* Fix wrong error being returned

* Call sync_all() on new collection

* Add ImportError

* Store logger in Backend, instead of creating one on demand

init_backend() accepts a Logger rather than a log file, to allow other
callers to customize the logger if they wish.

In the future we may want to explore using the tracing crate as an
alternative; it's a bit more ergonomic, as a logger doesn't need to be
passed around, and it plays more nicely with async code.

* Sync file contents prior to rename; sync folder after rename.

* Limit backup creation to once per 30 min

* Use zstd::stream::copy_decode

* Make importing abortable

* Don't revert if backup media is aborted

* Set throttle implicitly

* Change force flag to minimum_backup_interval

* Don't attempt to open folders on Windows

* Join last backup thread before starting new one

Also refactor.

* Disable auto sync and backup when restoring again

* Force backup on full download

* Include the reason why a media file import failed, and the file path

- Introduce a FileIoError that contains a string representation of
the underlying I/O error, and an associated path. There are a few
places in the code where we're currently manually including the filename
in a custom error message, and this is a step towards a more consistent
approach (but we may be better served with a more general approach in
the future similar to Anyhow's .context())
- Move the error message into importing.ftl, as it's a bit neater
when error messages live in the same file as the rest of the messages
associated with some functionality.

* Fix importing of media files

* Minor wording tweaks

* Save an allocation

I18n strings with replacements are already strings, so we can skip the
extra allocation. Not that it matters here at all.

* Terminate import if file missing from archive

If a third-party tool is creating invalid archives, the user should know
about it. This should be rare, so I did not attempt to make it
translatable.

* Skip multithreaded compression on small collections

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-03-07 15:11:31 +10:00
Damien Elmes
864904729b Fix an intermittent failure in one of the Rust unit tests 2022-03-07 14:18:17 +10:00
RumovZ
5426164ebf
Add regex tag search (#1707) 2022-03-04 18:43:27 +10:00
Damien Elmes
2c80bcb571 Update to latest rules_rust, which unblocks the tokio upgrade
Rust version has been pinned, as 1.59 gets stuck compiling pyo3_macros,
and appears to have other issues:
https://github.com/rust-lang/rust/issues/94390
2022-03-03 19:48:17 +10:00
RumovZ
508b5ab947
Remove top_deck_id arg in deck_tree() (#1702)
Counts don't propogate correctly anymore (#1678).
2022-03-02 15:30:32 +10:00
Damien Elmes
6941bccde4 Add support for proto3 optional scalars
Protobuf 3.15 introduced support for marking scalar fields like
uint32 as optional, and all of our tooling appears to support it
now. This allows us to use simple optional/null checks in our Rust/
TypeScript code, without having to resort to an inner message.

I had to apply a minor patch to protobufjs to get this working with
the json-module output; this has also been submitted upstream:
https://github.com/protobufjs/protobuf.js/pull/1693

I've modified CardStatsResponse as an example of the new syntax.

One thing to note: while the Rust and TypeScript bindings use optional/
null fields, as that is the norm in those languages, Google's Python
bindings are not very Pythonic. Referencing an optional field that is
missing will yield the default value, and a separate HasField() call
is required, eg:

```
>>> from anki.stats_pb2 import CardStatsResponse as R
... msg = R.FromString(b"")
... print(msg.first_review)
... print(msg.HasField("first_review"))
0
False
```
2022-02-27 19:42:06 +10:00
Damien Elmes
89bce6d829 Update Rust deps; pin Tokio
Tokio has had to be pinned, because the 1.17 release introduces
a dependency on windows_sys, which fails to build on Windows on
Bazel.

The issue appears to be the build script of a subcrate - it is using
CARGO_MANIFEST_DIR to update the linking path so windows.lib can be
found (it's contained in that crate), but the path is set incorrectly.

dfc25285a2/crates/targets/x86_64_msvc/build.rs

One way we might be able to work around it is to add to the link path
in our own build script.
2022-02-25 17:10:47 +10:00
Damien Elmes
d52c36e920 Make flag changes undoable again
The previous change in 1871b57663 failed
to consider the browser refreshing case, as reported here:
https://forums.ankiweb.net/t/anki-2-1-50-beta-3-4/17501/30

I previously attempted to solve this by having SetFlag skip the queue
rebuild, then mutating the captured mtimes in the queues. That didn't
work correctly when undoing, as the queue mutations weren't recorded.
This approach combines that attempt and the previous change: flag
setting is an undoable operation again, but does not change the card's
modification time, so it can be applied/undone without a queue build
being required. Instead of special-casing flag changes in the review
screen, we now just redraw the flag on changes.card, as any other card
op will have triggered a queue rebuild.
2022-02-25 15:06:07 +10:00
RumovZ
a4d61fe7d9
Original position (#1677)
* Replace Card.data with .original_position

* Use and update original position in v3

* Show original position in card info

* Revert restoring original position for now

* Fix pb card to/from pylib card

* Try original_position as the last pb field

* minor wording tweaks (dae)
2022-02-22 22:48:21 +10:00
RumovZ
131a94dd85
Config for burying interday learning cards (#1680)
* Add config for burying interday learning cards

* Expose bury interday learning config in GUI
2022-02-22 21:37:59 +10:00
Abdo
fd6a78e7cf
Add option to ignore accents in search by default (#1667)
* Add option to ignore accents in unqualified search

* Support ignoring accents in word boundary search

* Update ftl/core/preferences.ftl
2022-02-17 16:30:52 +10:00
Damien Elmes
1871b57663 don't put flag changes in the undo queue, and don't bump mtime
This is not ideal, but I struggled to come up with a better solution.

Background:

- The scheduler records the mtime of cards as it's building the queues,
and will throw an error in get_queued_cards() if the card on the DB
has a different mtime. This is to catch bugs - any operation that modifies
cards should be triggering a queue rebuild, or should adjust the queues
appropriately.
- The review screen skips the usual queue rebuild redraw, and directly
updates the flag icon. This is because a rebuild could cause a different
card to appear, or the answer side to switch back to the question side,
neither of which the user expects when they flag a card.

The current behaviour was broken: the queue rebuilding was still happening
on the backend, and the frontend was just failing to reflect it.

I initially tried to special-case Op::SetFlag, having it skip the queue
rebuild, and having set_card_flag() update the mtimes in the active
queue. But those mutations weren't captured by the undo log, so they
didn't get undone when undoing the set flag operation. We could perhaps
work around it by adding a separate undo entry to capture the mutation,
but it started to feel like it would be a pain to maintain moving forward.

By skipping the undo queue and retaining the same mtime, no queue
rebuild is required. Because we're setting usn, the cards will still
sync, but as mtime is not bumped, in the case of a conflict, an older
unsynced change from another client may revert the flag change.

Fixes https://forums.ankiweb.net/t/anki-2-1-50-beta-1-2/15608/145
2022-02-13 16:27:33 +10:00
RumovZ
6a21261c84
Round calculated Hard days (#1661)
* Round calculated Hard days

* clarify docstring (dae)
2022-02-13 13:37:52 +10:00
RumovZ
d55f080733
V3 parent limits (#1638)
* avoid repinning Rust deps by default

* add id_tree dependency

* Respect intermediate child limits in v3

* Test new behaviour of v3 counts

* Rework v3 queue building to respect parent limits

* Add missing did field to SQL query

* Fix `LimitTreeMap::is_exhausted()`

* Rework tree building logic

https://github.com/ankitects/anki/pull/1638#discussion_r798328734

* Add timer for build_queues()

* `is_exhausted()` -> `limit_reached()`

* Move context and limits into `QueueBuilder`

This allows for moving more logic into QueueBuilder, so less passing
around of arguments. Unfortunately, some tests will require additional
work to set up.

* Fix stop condition in new_cards_by_position

* Fix order gather order of new cards by deck

* Add scheduler/queue/builder/burying.rs

* Fix bad tree due to unsorted child decks

* Fix comment

* Fix `cap_new_to_review_rec()`

* Add test for new card gathering

* Always sort `child_decks()`

* Fix deck removal in `cap_new_to_review_rec()`

* Fix sibling ordering in new card gathering

* Remove limits for deck total count with children

* Add random gather order

* Remove bad sibling order handling

All routines ensure ascending order now.
Also do some other minor refactoring.

* Remove queue truncating

All routines stop now as soon as the root limit is reached.

* Move deck fetching into `QueueBuilder::new()`

* Rework new card gather and sort options

https://github.com/ankitects/anki/pull/1638#issuecomment-1032173013

* Disable new sort order choices ...

depending on set gather order.

* Use enum instead of numbers

* Ensure valid sort order setting

* Update new gather and sort order tooltips

* Warn about random insertion order with v3

* Revert "Add timer for build_queues()"

This reverts commit c9f5fc6ebe87953c17a0c842990b009b5596c69c.

* Update rslib/src/storage/card/mod.rs (dae)

* minor wording tweaks to the tooltips (dae)

+ move legacy strings to bottom
+ consistent capitalization (our leech action still needs fixing,
but that will require introducing a new 'suspend card' string as the
existing one is used elsewhere as well)
2022-02-10 09:55:43 +10:00
RumovZ
4909302147
Intersperse more evenly, mainly at start and end (#1651)
Closes #1645.
2022-02-09 12:45:37 +10:00
RumovZ
872b6df22a
Optimise searching in (all) fields (#1622)
* Avoid rebuilding regex in field search

* Special case search in all fields

* Don't repeat mid nodes in field search sql

Small speed gain for searches like `*:re:foo` and reduces the sql tree
depth if a lot of field names of the same notetype match.

* Add sql function to match fields with regex

* Optimise used field search algorithm

- Searching in all fields is a special case.
- Using native SQL comparison is preferred.
- For Regex, use newly added SQL function.

* Please clippy

* Avoid pyramid of doom

* nt_fields -> matched_fields

* Add tests for regex and all field searches

* minor tweaks for readability (dae)
2022-01-24 20:30:08 +10:00
RumovZ
d4e3cc7a0a
Remove into_node_list() (#1609) 2022-01-21 21:23:52 +10:00
RumovZ
9aca778a93
Backend Custom Study (#1600)
* Implement custom study on backend

* Switch frontend to backend custom study

* Skip typecheck for new pb classes

* Build tag search string on backend

Also fixes escaping of special characters in tag names.

* `cram.cards` -> `cram.card_limit`

* Assign more meaningful names in `TagLimit`

* Broaden rustfmt glob

* Use `invalid_input()` helper

* Assign `FilteredDeckForUpdate` to temp var

* Implement `SearchBuilder`

* Rewrite `custom_study()` with `SearchBuilder`

* Replace match macros with `SearchBuilder`

* Remove `into_nodes_list` & `concatenate_searches`
2022-01-20 14:25:22 +10:00
RumovZ
1dae2f42ff
Make note: and card: search require full match (#1606) 2022-01-20 11:57:16 +10:00
RumovZ
77b1546583
Remove redundant error mapping (#1593)
Helpful regex error in Find&Replace was obscured by generic
"Invalid input" error.
2022-01-16 13:46:27 +10:00
Damien Elmes
bbc191711d update Rust deps 2022-01-15 14:59:43 +10:00
Abdo
aaaea03c72
Fix new preview card's position being interpreted as a date (#1577)
* Fix new preview card's position being interpreted as a date

Can be reproduced by opening the Card Info screen of a new preview card
not answered yet.

* Update rslib/src/stats/card.rs
2022-01-06 19:52:54 +10:00
Damien Elmes
693672d7a3 remove extraneous to_vec() call 2021-12-24 13:09:19 +10:00
Damien Elmes
b7907741ec fix SQL formatting issue 2021-12-24 12:42:42 +10:00
Damien Elmes
8ed1fa7e99 fix interday learning cards not being buried
https://forums.ankiweb.net/t/2-1-49-mac-automatic-unbury-still-not-working/14799/43
2021-12-24 12:25:16 +10:00
Damien Elmes
4d431fb7af move linkchecker into separate crate
The feature-based approach didn't work with cargo-raze, leading
to ./update.py in cargo/ breaking.
2021-12-20 17:27:43 +10:00
RumovZ
3e0c9dc866
New TTS/AV tag handling (#1559)
* Add new `card_rendering` mod

Parses a text with av/tts tags and strips or extracts tags.

* Replace old `extract_av_tags` and `strip_av_tags`

... with new `card_rendering` mod

* ressource -> resource

* Add AV prettifier for use in browser table

* Accept String in av tag routines

... and avoid redundant writes if no changes need to be made.

* add benchmarking with criterion; make links test optional (dae)

cargo install cargo-criterion, then run ./bench.sh

* performance comparison: creating HashMap up front (dae)

the previous solution:

anki_tag_parse          time:   [1.8401 us 1.8437 us 1.8476 us]

this solution:

anki_tag_parse          time:   [2.2420 us 2.2447 us 2.2477 us]
                        change: [+21.477% +21.770% +22.066%] (p = 0.00 < 0.05)
                        Performance has regressed.

* Revert "performance comparison: creating HashMap up front" (dae)

This reverts commit f19126a2f15b729b825825a49283f63ab13474d0.

* add missing header

* Write error message if tts lang is missing

* `Tag` -> `Directive`
2021-12-17 19:04:42 +10:00
RumovZ
2221d0a520
Make Hard use current step's interval if it's not the first one (#1561)
* Make hard repeat the current step's interval in v3

Unless for the first step to avoid identical interval with Again.

* Make Hard repeat the current step's interval in v2

* Adjust test to new Hard behaviour
2021-12-16 22:02:13 +10:00
RumovZ
80ed94ed08
Cap steps (#1548)
* Fix steps being mistaken for seconds

* Cap steps at `u32::max` seconds

* Fix overflow of steps in Rust

* Prevent overflow of `IntervalKind`

* Prevent overflow in `revlod/mod.rs`

Also replace some `as` with `from` and `try_from` as is recommended to
highlight potential issues.

* Ensure v2 doesn't store overflowing revlog ivls

* Lower steps cap in deck options

Whereas large card intervals are converted to days, revlog intervals use
i32s to store large numbers of seconds.

* Format
2021-12-15 18:46:26 +10:00
Damien Elmes
8d90b6b061 run buildifier/buildozer to tidy up BUILD files 2021-12-14 09:18:24 +10:00
Damien Elmes
0c821859c5 check for missed dbg!() statements
This works in our main code, but doesn't catch dbg!() statements in
our build scripts.
2021-12-14 09:05:14 +10:00
Damien Elmes
afff4fc437 order by template after note id in filtered decks
This brings the behaviour a bit closer to the default ordering of new
cards when they are reset, and is better than an undefined template
order. But it's a stopgap solution, and in the long run, filtered decks
need a bit of a rethink with the improved ordering than v3 has brought.
2021-12-10 18:53:50 +10:00
Damien Elmes
d515939340 fix error when gathering new cards in reverse position
Also simplify order clause - with did and queue limited to a constant,
SQLite is smart enough to use the covering index for sorting by due.
2021-12-06 17:08:00 +10:00
Damien Elmes
c2ab0de34f update Rust deps 2021-12-03 20:35:53 +10:00
Damien Elmes
67ee6f9c0e update to Rust 1.57 + latest rules_rust 2021-12-03 20:35:52 +10:00
Damien Elmes
8de3eaea65 fix Clippy lints in Rust 1.57 2021-12-03 19:53:37 +10:00
Damien Elmes
d7f5555642 deck_tree() could be made to unbury the current day's cards
When provided with a future timestamp to estimate the next day's cards,
it lead to the current day's buried cards being unburied.

Fixes
https://forums.ankiweb.net/t/inconsistent-card-counts-when-syncing/15496
2021-12-02 21:07:21 +10:00
Damien Elmes
9daa0c9acb fix new cards not appearing in correct order in v3
This was broken by an SQLite upgrade - previously we received the rows
in ix_cards_sched order, but recent versions use a table scan for that
query when the order is unspecified. Solved by being explicit about the
order we expect results to arrive.

https://forums.ankiweb.net/t/skipping-new-cards/15410
2021-12-02 17:20:48 +10:00
Matthias Metelka
68092082f2
Change Notetype UI Rework (#1499)
* Enable access to old notetype name

* Set minimum height for ChangeNotetypeDialog

* Add bootstrap icons to change-notetype

* Move alert up and make it collapsible

* Tweak some CSS

- Add variables --sticky-bg and --sticky-border to StickyContainer
- Tweak base.css

* Add translatable string "(Nothing)"

* Rework ChangeNotetype screen

* Initially load option at newIndex and remaining options on focus

Optimization for big notetypes:
Should increase efficiency from O(n²) to O(n). Test on notetype with 500 templates shows significant improvement in load time (~10s down to ~1s).

* Try to satisfy rust test

* Change arrow direction depending on reading direction

+ add 0.5em top padding to main

* Create Alert.svelte

* Introduce CSS variable --pane-bg

* Revert "Initially load option at newIndex and remaining options on focus"

This reverts commit f42beee45c27dba9433d76217fb583b117fb5231.

* Final cleanup

* Refine padding/gutter
2021-11-24 12:09:55 +10:00
Henrik Giesel
2778b9220c
Mathjax editor improvements (#1502)
* Remove unnecessary stopPropagation of mathjax-overlay events

* Use CodeMirror component for MathjaxHandle

* Refactor ResizeObserver code in MathjaxHandle

* Wrap setRange in CodeMirror in try/catch

* Add Mathjax Editor bottom margin

* Add custom Enter and Shift+Enter shortcuts for the MathjaxHandle

* Format

* Move placeCaretAfter to domlib

* Move focus back to field after editing Mathjax

* Put Cursor after Mathjax after accepting

* Add delete button for Mathjax

* Change border color of mathjax menu

* Refactor into MathjaxMenu

* Put caretKeyword in variable

* Use one ResizeObserver for all Mathjax images

* Add minmimum width for Mathjax editor

* is still smaller than minimal window width

* Add bazel directories to .prettierignore and format from root

* exclude ftl/usage (dae)

the json files that live there are output from our tooling, and
formatting them means an extra step each time we want to update them

also exclude .mypy_cache, which is output by scripts/mypy*

* minor ftl tweak: newline -> new line  (dae)
2021-11-23 10:27:32 +10:00
Damien Elmes
11042a16ce Fix latest Rusqlite on Windows
https://github.com/bazelbuild/rules_rust/pull/1025
2021-11-19 10:44:02 +10:00
Damien Elmes
8b3d964866 roll rusqlite back due to Windows breakage 2021-11-18 21:39:23 +10:00
Damien Elmes
d31dccc388 update Rust deps 2021-11-18 20:58:41 +10:00
Damien Elmes
26123ce229 update to edition 2021 2021-11-18 20:51:10 +10:00
RumovZ
72b08cb7d5
Smooth fuzz (#1493)
* Add separate `fuzz.rs`

* Switch to a smoother fuzz calculation
2021-11-17 07:23:19 +10:00
Damien Elmes
13382f7f87 avoid fuzzing until interval reaches 3 days
e5e47a31fe (r748827327)

+ switched assert_lower_middle_upper to a macro, so that when it fails,
the reported line number is the original call site, instead of one inside
the helper function
2021-11-15 15:48:58 +10:00
RumovZ
b1142c12d9
Fix constrained_fuzz_bounds() (#1490)
... for cases where the entire fuzz range is above `maximum`.
Also improve hanling if the entire range is below `minimum` and
readability.
2021-11-15 15:41:43 +10:00
Damien Elmes
311e70822d fix a link anchor 2021-11-14 11:44:54 +10:00
Damien Elmes
5f05eeb922 unbury when refreshing queues
While we already unbury when refreshing the deck list, if the user
resumes study on a new day without refreshing the deck list, burying
could end up being delayed.

Possible fix for https://forums.ankiweb.net/t/buried-cards-in-ankimobile-beta-20081-3/14753/3
2021-11-14 10:06:47 +10:00
Damien Elmes
044d253306 fix underflow in fuzz code, leading to large intervals
https://forums.ankiweb.net/t/buried-cards-in-ankimobile-beta-20081-3/14753
2021-11-14 09:17:37 +10:00
RumovZ
0efa3f944f
Garbage collect unused Fluent strings (#1482)
* Canonify import of i18n module

Should always be imported as `tr`, or `tr2` if there is a name collision
(Svelte).

* Add helper for garbage collecting ftl strings

Also add a serializer for ftl asts.

* Add helper for filter-mapping `DirEntry`s

* Fix `i18n_helpers/BUILD.bazel`

* run cargo-raze

* Refactor `garbage_collection.rs`

- Improve helper for file iterating
- Remove unused terms as well
- Fix issue with checking for nested messages by switching to a regex-
based approach (which runs before deleting)
- Some more refactorings and lint fixes

* Fix lints in `serialize.rs`

* Write json pretty and sorted

* Update `serialize.rs` and fix header

* Fix doc and remove `dbg!`

* Add binaries for ftl garbage collection

Also relax type constraints and strip debug tests.

* add rust_binary targets for i18n helpers (dae)

* add scripts to update desktop usage/garbage collect (dae)

Since we've already diverged from 2.1.49, we won't gain anything
from generating a stable json just yet. But once 2.1.50 is released,
we should run 'ftl/update-desktop-usage.sh stable'.

* add keys from AnkiMobile (dae)

* Mention caveats in `remove-unused.sh`
2021-11-12 18:19:01 +10:00
Damien Elmes
cb07b232d8 add a builder for Collection 2021-11-06 14:43:41 +10:00
RumovZ
283776d8e7
Rework v3 fuzzing (#1474)
* Remove flooring in v3 scheduler code

It is no longer supposed to be an exact port of the old Python code.

* Rework v3 fuzzing

https://github.com/ankitects/anki/issues/1416#issuecomment-958208149

* Ensure length of fuzz range is larger than 1

Only for new intervals larger than 1 and respecting max review interval.

* add the beginnings of a unit test

* Clarify `fuzz_factor` doc string

* Fix Python tests for 2021 scheduler

* Fix fuzz test

1.0 is not a valid fuzz factor.

* Add tests for fuzzing in Rust

* Use range notation in fuzz factor doc

* Strip redundant tests
2021-11-06 10:39:24 +10:00
Matthias Metelka
371f731e30
Editor Field Descriptions (#1476)
* Add description input to fields dialog

QLineEdit seems like the best option, as it saves space and motivates users to keep their descriptions concise.

* Add setDescriptions to note initialization script

Went for the extra function instead of including it in setFields to prevent potential add-on breakages.

* Add tooltip next to field name if description is set

* Refactor code according to suggestions

Set default tooltip placement to right instead of bottom

Use .get() for fld["description"]

Fix tab order in fields dialog

Swap out abbreviation "desc" for full length name to keep consistency

* Update Protobuf and Rust for description

Add description to notetypes.proto and schema11

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>

* Fix tooltips not updating with description

Remove redundant variable tooltipOptions

Update previousTooltip within reactive function

* Move LabelDescription out of LabelName

Co-authored-by: Henrik Giesel <hgiesel@users.noreply.github.com>

* Decrease icon size and fix alignment

Co-Authored-By: Henrik Giesel <hengiesel@gmail.com>

* the new key needs to be cleared from fields, not the notetype itself

Co-authored-by: RumovZ <RumovZ@users.noreply.github.com>
Co-authored-by: Henrik Giesel <hengiesel@gmail.com>
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2021-11-06 09:42:48 +10:00
Damien Elmes
3843a4509e fix sort order toggling
We're getting an enum instead of an int in Qt6

normal/reversed have been renamed to ascending/descending; no add-ons
appear to be using the old versions.
2021-10-30 09:14:43 +10:00
RumovZ
16ad0137f7
Use backend for getting deck children (#1465)
* Add backend routine for child deck names and ids

* Use backend for getting deck children

Instead of flawed case-sensitive Python logic.
2021-10-29 19:43:17 +10:00
Damien Elmes
0e00c4a461 fix new cards not being correctly limited
https://forums.ankiweb.net/t/ios-beta-20080-2-more-new-cards-after-review-limit-is-met/13728/10
2021-10-29 12:12:34 +10:00
RumovZ
7e23c9fe4f
Fix quotation of "and" and "or" in search (#1463) 2021-10-28 19:23:56 +10:00
RumovZ
3ab9712c18
Stop trimming filename references before encoding (#1462)
Closes  #1430
2021-10-28 19:22:51 +10:00
Damien Elmes
e30411e0b2 update Rust deps with cargo-raze 0.13.0 2021-10-26 08:16:40 +10:00
RumovZ
3cdb3d72c1
Do not bury suspended cards (#1447)
* Skip burying for suspended cards

* Inform about number of buried cards
2021-10-23 11:04:26 +10:00
RumovZ
1c9b5a2e83
Card info cleanup (#1446)
* Cast proto interface to type ...

... instead of using non-null assertions in Revlog.svelte.

* Remove OptionalInt32 and OptionalUInt32
2021-10-23 11:00:43 +10:00
RumovZ
56d8402f89
Fix underflow of learning count (#1444)
`counts.learning` includes interday learning cards, so it is not
suitable to determine how many cards from the (intraday!) learning queue
are already included in the learning count when updating it.
2021-10-22 20:58:06 +10:00
Damien Elmes
61f3b71664 fix a clippy lint in 1.56 2021-10-22 12:03:54 +10:00
Damien Elmes
3d416b4a68 convert sql_format to ts_project; move into separate folder 2021-10-18 19:39:45 +10:00
Damien Elmes
a29bd7c9f0 unbury when rendering deck list
We were previously relying on the sched_timing_today() call in the
backend, but v3 doesn't call it, leading to cards remaining buried.
2021-10-16 21:11:09 +10:00
RumovZ
3672b0fe73
Switch CardInfoDialog to ts page (#1414)
* Only collect card stats on the backend ...

... instead of rendering an HTML string using askama.

* Add ts page Card Info

* Update test for new `col.card_stats()`

* Remove obsolete CardStats code

* Use new ts page in `CardInfoDialog`

* Align start and end instead of left and right

Curiously, `text-align: start` does not work for `th` tags if assigned
via classes.

* Adopt ts refactorings after rebase

#1405 and #1409

* Clean up `ts/card-info/BUILD.bazel`

* Port card info logic from Rust to TS

* Move repeated field to the top

https://github.com/ankitects/anki/pull/1414#discussion_r725402730

* Convert pseudo classes to interfaces

* CardInfoPage -> CardInfo

* Make revlog in card info optional

* Add legacy support for old card stats

* Check for undefined instead of falsy

* Make Revlog separate component

* drop askama dependency (dae)

* Fix nightmode for legacy card stats
2021-10-14 19:22:47 +10:00
Damien Elmes
ca32a46594 handle AnkiDroid's two element new intervals
https://github.com/ankidroid/Anki-Android/issues/8889
2021-10-07 23:15:51 +10:00
Damien Elmes
944b064e54 update Rust deps 2021-10-02 20:42:03 +10:00
Damien Elmes
8e56853c50 fix a clippy lint in latest Rust 2021-10-02 14:13:49 +10:00
Damien Elmes
0bb273a0ed replace the old stripHTML() methods with the backend implementation
Python's regex engine performs pathologically on regexes like
'<!--.*?-->' when fed a large string of repeating '<!--' clauses.
Thanks to JaimeSlome / security@huntr.dev for the report; closes #1380.

Solved by switching to the Rust implementation, which does not suffer
from this issue.

entsToText(), minimizeHTML(), and the old regex constants have been
removed; they do not appear to be used by any add-ons.
2021-10-01 23:15:45 +10:00
Damien Elmes
a174c41801
Merge pull request #1397 from RumovZ/column-tooltips
Add tooltips for some browser columns
2021-10-01 19:12:59 +10:00
Damien Elmes
3940b0587d avoid extra allocation in replacement 2021-10-01 14:43:38 +10:00
RumovZ
ee2ecd0700 Add tooltips for some browser columns 2021-09-30 13:15:09 +02:00
abdo
018594c5fb Strip isolation chars from filenames in Check Media screen 2021-09-27 22:15:37 +03:00
Damien Elmes
9daf037c0b
Merge pull request #1264 from hgiesel/tageditor2
Svelte Tag Editor v2
2021-09-15 20:21:16 +10:00
Damien Elmes
70a367137d handle filtered case when repositioning 2021-09-13 14:57:41 +10:00
Damien Elmes
6296e93c21 check for invalid collections in full_upload() 2021-09-13 11:50:13 +10:00
Damien Elmes
6fb6ffa1d7 tentative fix for learning count underflow
The 'avoid showing learning card twice' logic is now only applied
when the next learning card was already due to be shown. This'll mean
there will be cases where a learning card does get shown twice near
the end, but it makes the behaviour easier to reason about, for both
us and end users.
2021-09-13 11:08:55 +10:00
Damien Elmes
c9d3e5462e bikeshedding: amount -> match_limit
+ convert from u32 in backend method
2021-09-12 11:57:49 +10:00
Henrik Giesel
e85c93f3e7 Cap tag suggestions at 500 2021-09-10 01:13:50 +02:00
Damien Elmes
4da1c77220 add basic tag completion to backend
Matches should arrive in alphabetical order. Currently results are not
capped (JS should be able to handle ~1k tags without too much hassle),
and no reordering based on match location is done. Matches are substring
based, and multiple can be provided, eg "foo::bar" will match
"foof::baz::abbar".

This is not hooked up properly on the frontend at the moment -
updateSuggestions() seems to be missing the most recently typed character,
and is not updating the list of completions half the time.
2021-09-09 15:38:08 +02:00
Damien Elmes
a961013961 fix case of fields table 2021-09-07 10:22:22 +10:00
Damien Elmes
9a27e44ff8 don't panic when invalid note id provided to change notetype
https://forums.ankiweb.net/t/fatal-error-while-updating-deck/12743/2
2021-08-29 14:00:55 +10:00
Damien Elmes
dffa38c479 expose uncapped counts and total in tree node
In the future this information could be revealed on hover, to make it
clearer how limits are being applied.
2021-08-22 15:32:46 +10:00
Damien Elmes
05124eecf7 adding options to sort by ascending/descending ease 2021-08-22 15:32:46 +10:00
Damien Elmes
62223499c4 fix errors when undoing/redoing after a queue-invalidating operation
There were a few issues going on here:

- If some operation had invalidated the queues, they were subsequently
recreated with a call to .get_queues() in the undo handling code. This
could happen after the changes to the card had already been reverted,
leading to a queue state that didn't match our expectations.
- More generally, it's not safe to assume our mutations will apply
cleanly after the queue has been rebuilt. The next card will vary
depending on the number of remaining cards when interspersing cards of
different types, and a queue-invalidating operation will have changed
the learning cutoff.

So rather than rebuilding the queues on demand, we now check that they
already exist, and were created at the time we expect. If not, we
invalidate them and skip applying the mutations, and a subsequent
refresh of the UI should rebuild the queues correctly.

As part of this change, the cutoff snapshot has been moved into the
normal answer update object.

One possible downside here is that adding a note during review may cause
a newly due learning card to appear when undoing a different review.
If this proves to be a problem, we could potentially note down the
learning cutoff and apply it when queues are rebuilt later.
2021-08-22 15:32:46 +10:00
Damien Elmes
b9402b5c47 support limiting interday learning cards by review limit again
Context: https://forums.ankiweb.net/t/more-cards-today-question-about-v3/12400/10

Previously, interday learning cards and reviews were gathered at the
same time in v3, with the review limit being applied to both of them. The
order cards were gathered in would change the ratio of gathered learning
cards and reviews, but as they were displayed together in a single count,
a changing ratio was not apparent, and no special handling was required
by the deck tree code.

Showing interday learning cards in the learning count, while still
applying a review limit to them, makes things more complicated, as
a changing ratio will result in different counts. The deck tree code
is not able to know which order cards will appear in, so without changes,
we would have had a situation where the deck list may show different counts
to those seen when clicking on a deck.

One way to solve this would have been to introduce a separate limit for
interday learning cards. But this would have meant users needed to
juggle two different limits, instead of having a single one that controls
total number of (non-intraday) cards shown.

Instead, the scheduler now fetches interday cards prior to reviews -
the rationale for that order is that learning cards tend to be more
fragile/urgent than reviews. The option to show learning cards
before/after/mixed with reviews still exists, but it applies only after
cards have been capped to the daily limit.

To ensure the deck tree code matches the counts the scheduler gives,
it too applies limits to interday learning cards first, and reviews
afterwards.
2021-08-22 15:32:46 +10:00
Damien Elmes
6977b6ab6f fix new clippy warning 2021-08-22 15:32:46 +10:00
Damien Elmes
131c8b72f8 add options to v3 to preserve new card gather order
Allows cards to be presented in deck order when gather priority is set
to 'deck'.
2021-08-20 12:03:32 +10:00
Damien Elmes
53fe7e574e handle ampersand entities in image filenames
In the old HTML editor, filenames were % escaped before feeding them to
beautifulsoup, causing bare ampersands to be left alone. The new HTML
editor reads content from the DOM, where a bare ampersand has been
transformed into an &amp;, and that gets saved back into the field,
so the media check now needs to deal with it for images as well.

https://forums.ankiweb.net/t/causing-problems-with-image-names/12171
2021-08-19 23:43:40 +10:00
Damien Elmes
48c121e4f3 filtered decks w/ scheduling disabled in v3 now log reviews 2021-08-19 20:25:29 +10:00
Damien Elmes
8830d33826 revert some interday learning changes in v3
Interday learning cards are now counted in the learning count again,
and are no longer subject to the daily review limit.

The thinking behind the original change was that interday learning cards
are scheduled more like reviews, and counting them in the review count
would allow the learning count to focus on intraday learning - the red
number reflecting the fact that they are the most fragile memories. And
counting them together made it practical to apply the review limit
to both at once.

Since the release, there have been a number of users expecting to see
interday learning cards included in the learning count (the latest being
https://forums.ankiweb.net/t/feedback-and-a-feature-adjustment-request-for-2-1-45/12308),
and a good argument can be made for that too - they are, after all, listed
in the learning steps, and do tend to be harder than reviews. Short of
introducing another count to keep track of interday and intraday learning
separately, moving back to the old behaviour seems like the best move.

This also means it is not really practical to apply the review limit to
interday learning cards anymore, as the limit would be split between two
different numbers, and how much each number is capped would depend on
the order cards are introduced. The scheduler could figure this out, but
the deck list code does not know card order, and would need significant
changes to be able to produce numbers that matched the scheduler. And
even if we ignore implementation complexities, I think it would be more
difficult for users to reason about - the influence of the review limit
on new cards is confusing enough as it is.
2021-08-19 16:40:12 +10:00
Damien Elmes
272a610832 ensure interday learning cards update deck limits
https://forums.ankiweb.net/t/more-cards-today-question-about-v3/12400
2021-08-19 12:50:11 +10:00
Damien Elmes
d333d0da74 allow adjusting tag case in rename
https://forums.ankiweb.net/t/2-1-46-renaming-tags-to-tags-not-working/12426
2021-08-19 11:46:01 +10:00
Damien Elmes
1f4d54efda allow repositioning of new cards while suspended
https://forums.ankiweb.net/t/no-longer-can-reposition-a-suspended-card/12241
2021-08-15 15:06:25 +10:00
Damien Elmes
187944615e check both queues when popping answered card
If multiple answers are applied in succession, a newly-due learning card
may be in front of the next main entry.
2021-08-09 18:29:04 +10:00
abdo
52fac0557a Fix double quotes being escaped twice in dupe search
maybe_quote() already escapes double quotes.
2021-08-09 05:05:20 +03:00
Damien Elmes
6548ebb516 fall back on default deck in v3 queue gather as well
Missed in the previous change.

https://forums.ankiweb.net/t/error-when-trying-to-choose-deck-in-new-anki/11829/11
2021-08-09 11:04:37 +10:00
Damien Elmes
94913ec23f fallback on default deck in congrats screen
https://forums.ankiweb.net/t/error-when-trying-to-choose-deck-in-new-anki/11829
2021-08-04 12:57:46 +10:00
Damien Elmes
746b19e38c fix undoing last learning card corner case 2021-08-04 10:29:01 +10:00
Damien Elmes
7ba35b7249 support updating multiple notes in one transaction/undo op 2021-08-02 17:07:26 +10:00
Damien Elmes
88a3fd8d7b support updating multiple cards in one transaction/undo op 2021-08-02 16:59:02 +10:00
Damien Elmes
cc65196b0a revert to defaults when schema 11 deck config can not be read
Allows decks with missing/null values like ivlFct to be opened.

https://forums.ankiweb.net/t/i-cant-access-ankiweb/11814
2021-08-02 16:27:48 +10:00
Damien Elmes
070f57fcc5 don't hide learning count on congrats screen when learning is overdue
The v3 scheduler will delay the final card from being shown twice in
a row, but the overdue case was being treated the same as the no-learning
case, leading to the message being hidden.
2021-08-02 15:57:09 +10:00
Damien Elmes
3cec1a35dc update learning cutoff when counts are zero 2021-08-02 15:03:14 +10:00
Damien Elmes
94c7c3282b fix learning cutoff not updating during review 2021-08-02 15:03:14 +10:00
Damien Elmes
cf731c6bad fix v3 not honoring initial ease factor
Closes #1317
2021-07-31 14:57:04 +10:00
RumovZ
d090bd7e21 Use existing 'no such field' tr string 2021-07-29 08:30:49 +02:00
RumovZ
d6e5f3c67c Allow duplicate templates if {{Card}} is used 2021-07-28 22:32:38 +02:00
RumovZ
c0dd769090 Skip new notetype checks when importing apkg 2021-07-28 21:46:51 +02:00
RumovZ
be1b524396 Find template errors hidden by conditionals 2021-07-28 12:13:14 +02:00
RumovZ
4d7fcf585a Check for invalid conditionals on templates 2021-07-28 11:53:31 +02:00
Damien Elmes
6fe6570da3 fix reqwest build on Linux again 2021-07-24 10:18:09 +10:00
Damien Elmes
d73852f272 use separate integration test for links
If we run into issues with unreliable network connections in the future,
we'll be able to mark the test as flaky so Bazel can retry it multiple
times.
2021-07-24 10:12:25 +10:00
Damien Elmes
56545db339
Merge pull request #1308 from RumovZ/check-help
Build and check help links on the backend
2021-07-24 09:37:23 +10:00
RumovZ
07752f43cd Fix linkcheck result message 2021-07-23 19:40:51 +02:00
Damien Elmes
b1dedb1b1f disable link check outside CI 2021-07-23 20:22:32 +10:00
Damien Elmes
0578729468 be explicit about .html suffix 2021-07-23 20:22:32 +10:00
Damien Elmes
1ecda8cf4d fix clippy lints 2021-07-23 20:22:32 +10:00
Damien Elmes
379694915e add linkcheck to Bazel 2021-07-23 20:22:32 +10:00
RumovZ
503bdb8c22 Add check to ensure parsed URL equals link 2021-07-23 11:44:19 +02:00
RumovZ
6ef52726da Improve error message for unknown fragment 2021-07-23 10:32:29 +02:00
RumovZ
f2f19e8b45 Remove native HelpPage enum
Also remove oneof from pb enum and handle strs in Python.
2021-07-22 16:32:49 +02:00
RumovZ
55e1f178ef Add links.proto and backend module 2021-07-22 10:05:38 +02:00
RumovZ
80264c3b56 Add links.rs for checking and building links 2021-07-22 10:03:03 +02:00
Damien Elmes
5e10087aae handle notes with missing cards in browser
https://forums.ankiweb.net/t/2-1-45-release-candidate/11362/30
2021-07-22 14:58:57 +10:00
Damien Elmes
c944dd048e strip invalid Unicode chars in media check 2021-07-17 18:30:19 +10:00
Damien Elmes
2f434dd74d fix comment + copy/paste error 2021-07-17 09:02:14 +10:00
Aleksa Sarai
d7caafe91c
scheduler: use deck config's initial ease in set_due_date
Previously we would just use 250% ease for any new card that had no
pre-configured ease, but this will result in decks that have
non-standard ease values to have "set due date" cards in them that don't
match. In order to make this somewhat more efficient, we cache
deckid->ease lookups during this operation.

Ref: <https://forums.ankiweb.net/t/set-due-date-doesnt-obey-default-ease-factor/9184>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2021-07-16 12:33:15 +10:00
Damien Elmes
96e6e216a7
Merge pull request #1293 from RumovZ/find-and-replace
Find & Replace
2021-07-16 10:47:16 +10:00
Damien Elmes
bf507cca98 move from Python's URI escaping to IRI escaping in Rust
Should make non-Latin text readable in the HTML editor, without the
breakages reverted in the previous change.
2021-07-16 10:38:00 +10:00
RumovZ
a5193339e7 Rework Find & Replace dialog:
- Add option to affect whole collection
- Allow to open without selection
- Add parameter for presetting field
2021-07-13 16:33:45 +02:00
RumovZ
b148b7b5e0 Fix find & replace for notes without 'field_name'
Distinguish between no 'field_name' passed and 'field_name' not on note.
2021-07-13 16:21:13 +02:00
Damien Elmes
f649f6c92a minor tidyup in protobuf build script 2021-07-12 16:15:38 +10:00
Damien Elmes
9c354f5e6b drop the cloze-in-basic check for now
Unfortunately a popular note taking tool has been misusing cloze
markers in its deck exports. We may want to add this back in the
future, but we'll probably want to start by warning users, to give
people time to adjust.
2021-07-11 20:31:02 +10:00
Damien Elmes
0e7411188b configs.proto plural workaround no longer necessary 2021-07-11 19:35:18 +10:00
Damien Elmes
e61a611af7 rename Config in protobuf to avoid conflict with module name
+ use the enum directly, instead of wrapping it in an object

Python code retains the old "Config" name.
2021-07-11 19:27:08 +10:00
Damien Elmes
185e9acd22 split out remaining tags, stats, media and rendering 2021-07-10 23:16:18 +10:00
Damien Elmes
35b059ecdb split out sync, search, scheduler & config 2021-07-10 21:33:12 +10:00
Damien Elmes
9e0a295ab9 split out decks, deckconfig, notes, notetypes 2021-07-10 20:44:22 +10:00
Damien Elmes
18851ace47 split out cards and collection 2021-07-10 19:52:31 +10:00
Damien Elmes
616db33c0e refactor protobuf handling for split/import
In order to split backend.proto into a more manageable size, the protobuf
handling needed to be updated. This took more time than I would have
liked, as each language handles protobuf differently:

- The Python Protobuf code ignores "package" directives, and relies
solely on how the files are laid out on disk. While it would have been
nice to keep the generated files in a private subpackage, Protobuf gets
confused if the files are located in a location that does not match
their original .proto layout, so the old approach of storing them in
_backend/ will not work. They now clutter up pylib/anki instead. I'm
rather annoyed by that, but alternatives seem to be having to add an extra
level to the Protobuf path, making the other languages suffer, or trying
to hack around the issue by munging sys.modules.
- Protobufjs fails to expose packages if they don't start with a capital
letter, despite the fact that lowercase packages are the norm in most
languages :-( This required a patch to fix.
- Rust was the easiest, as Prost is relatively straightforward compared
to Google's tools.

The Protobuf files are now stored in /proto/anki, with a separate package
for each file. I've split backend.proto into a few files as a test, but
the majority of that work is still to come.

The Python Protobuf building is a bit of a hack at the moment, hard-coding
"proto" as the top level folder, but it seems to get the job done for now.

Also changed the workspace name, as there seems to be a number of Bazel
repos moving away from the more awkward reverse DNS naming style.
2021-07-10 19:17:05 +10:00
Damien Elmes
80b98e0db8 move protobuf into separate folder in preparation for multiple files 2021-07-09 21:02:40 +10:00
RumovZ
5067622751 Add pylib/browser.py for literal config keys
Also, remove config bools for sort order.
2021-07-05 12:44:48 +02:00
Henrik Giesel
aacf20531e Add NOTETYPE_FIELD sidebar items 2021-06-30 21:02:23 +02:00
Damien Elmes
79ec9b14b1 zero out graves before comparison, instead of at creation
Will allow us to turn the check back on in the future without a client
update
2021-06-30 10:08:52 +10:00
Damien Elmes
3ee18fb854 ignore graves in sanity check 2021-06-30 09:31:02 +10:00
Damien Elmes
576b141e2b add back in missing check for upload size
Compression now happens up-front, so we can tell in advance if the
upload size has been exceeded.
2021-06-29 10:50:03 +10:00
Damien Elmes
e3c9808b79 catch invalid card ordinals in DB check
https://forums.ankiweb.net/t/error-corrupted-note/10976
2021-06-29 10:50:03 +10:00
Damien Elmes
2a93355824 PEP8 cards.py 2021-06-27 12:12:23 +10:00
Damien Elmes
ca0374782e update to latest rusqlite 2021-06-25 16:22:21 +10:00
Damien Elmes
59e17950ad update most rust deps; skip rusqlite 2021-06-25 15:35:25 +10:00
Damien Elmes
73d9391f64 update undo skipping; exclude deck/tag expand/collapse
Instead of calling a method inside the transaction body, routines
can now pass Op::SkipUndo if they wish the changes to be discarded
at the end of the transaction. The advantage of doing it this way is
that the list of changes can still be returned, allowing the sync
indicator to update immediately.

Closes #1252
2021-06-25 09:16:15 +10:00
Damien Elmes
b693bdbf28 cope with collections where latexsvg is not a bool 2021-06-24 15:29:21 +10:00
Damien Elmes
47284867f9 report object type where json decode fails 2021-06-24 15:28:24 +10:00
Damien Elmes
0b697f5161 expose scheduler js option in deck config 2021-06-24 14:38:10 +10:00
Damien Elmes
114eec6585 allow customization of default search text
https://forums.ankiweb.net/t/whats-the-default-search-when-browsing-now-also-how-do-i-choose-to-browse-current-deck/8922
2021-06-24 11:24:30 +10:00
Damien Elmes
b392020798 fix clippy lints for latest Rust 2021-06-21 13:09:36 +10:00
Damien Elmes
c79f8ba88f in/out -> request/response
The saved characters weren't worth the increased difficulty when
reading, and the fact that we were deviating from protobuf norms.
2021-06-20 15:49:20 +10:00
Damien Elmes
a58826941f add rustls-native-certs
closes #1241
2021-06-18 18:15:41 +10:00
Damien Elmes
2e53dc63c8
Merge pull request #1230 from RumovZ/fields-check
Check for misplaced or missing clozes when adding and in the editor
2021-06-17 21:26:16 +10:00
RumovZ
87a50f22e6 Assign dupe error the lowest precedence 2021-06-16 11:42:40 +02:00
Henrik Giesel
0f658de702 Add escape_anki_wildcards_for_search_node 2021-06-16 09:25:27 +02:00
Henrik Giesel
b8142d33c8 Revert "Remove special treatment of tag:none"
This reverts commit 95285ef66d02d9d3f0d560f70514b16e8a2f7bf5.
2021-06-16 17:19:21 +10:00
Henrik Giesel
b19e07c135 Remove special treatment of tag:none 2021-06-16 17:19:21 +10:00
Henrik Giesel
c5faf39d7c Make Browser root nodes use "_*" uniformly 2021-06-16 17:19:21 +10:00
Damien Elmes
ecdd668f9a update Rust deps 2021-06-16 16:10:57 +10:00
RumovZ
d7340d3f07
Fix missing hyphen escape when normalizing search (#1233)
* Fix missing hyphen escape when normalizing search

* Add some more tests for normalization

* Use char to satisfy clippy
2021-06-15 10:02:39 +10:00
Damien Elmes
220fca9a1d handle <br/> when rendering a single line
+ case-insensitive matching

https://forums.ankiweb.net/t/html-editor-modifies-note-when-a-field-with-break-tags-is-opened/10772
2021-06-14 13:05:48 +10:00
RumovZ
aeedb4dc11 Add check for out-of-place/missing clozes 2021-06-12 10:02:21 +02:00
Damien Elmes
61e86cc29d new change notetype implementation for the frontend
- changes can now be undone
- the same field can now be mapped to multiple target fields, allowing
fields to be cloned
- the old Qt dialog has been removed
- the old col.models.change() API calls the new code, to avoid
breaking existing consumers. It requires the field map to always
be passed in, but that appears to have been the common case.
- closes #1175
2021-06-10 22:19:24 +10:00
Damien Elmes
40dc2e217c fix spurious text in header injected by rust analyzer 2021-06-10 09:06:37 +10:00
Damien Elmes
1f2567e04c add notetype changing to backend 2021-06-09 20:56:52 +10:00
Damien Elmes
410660990e add LIFO sorting options for new cards 2021-06-08 14:01:46 +10:00
Damien Elmes
1479957538 fix note changes triggering a queue rebuild 2021-06-08 12:09:35 +10:00
Damien Elmes
4325f867a7
Merge pull request #1217 from RumovZ/flag-ftl
Mark search-invalid-flag as a new string
2021-06-03 10:50:11 +10:00
RumovZ
f33bd4ca11 search-invalid-flag -> search-invalid-flag-2 2021-06-02 21:49:33 +02:00
Damien Elmes
2903848c62 handle duplicate ids in values when upgrading deck config
Multiple configs with the same inner id would lead to errors like the
following when trying to open the collection:

DeckConfigInner.interval_multiplier: invalid wire type: StartGroup (expected ThirtyTwoBit)
2021-06-02 17:12:59 +10:00
Damien Elmes
578b1b0552
Merge pull request #1213 from RumovZ/new-flags
Add pink, turquoise and purple flags
2021-06-02 11:22:26 +10:00
RumovZ
5e7a47fdc7 Rename violet to pink 2021-06-01 18:14:43 +02:00
Damien Elmes
50961a9196 push review randomizing into SQL
This makes the review backlog case more expensive, since we end up
shuffling items outside the daily limit, but for the common case it's
about the same speed, and it means we don't need two separate sorting
steps. New cards remain handled the same way, since a backlog
is common there.

Also ensures that interday learning cards honor the deck sorting, and
that the non-default sort orders shuffle at the end.
2021-06-01 14:50:35 +10:00
Damien Elmes
562787bce1 add options to sort reviews by deck
https://forums.ankiweb.net/t/is-studying-subdeck-by-subdeck-broken-in-2-1-44-mac/10458/2
2021-06-01 13:22:39 +10:00
Damien Elmes
775beda6ce fix subdeck order not being preserved by active_decks in v3 2021-06-01 11:40:25 +10:00