langfingaz/anki
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,032 Commits 1 Branch 1 Tag 37 MiB
3f403040db
Commit Graph

640 Commits

Author SHA1 Message Date
Damien Elmes
70cc1699a6 rewrite DB check 
- notes with wrong field count are now recovered instead of
being deleted
- notes with missing note types are now recovered
- notes with missing cards are now recovered
- recover_missing_deck() still needs implementing
- checks required
 2020-05-12 21:13:34 +10:00
Damien Elmes
4c9b6be832 fetch notes in nid order 
May reduce the amount of seeking on conventional disks. Chunking and
fetching batches of notes at once would improve things further, at the
cost of more memory use.
 2020-05-12 21:13:34 +10:00
Damien Elmes
2413f286b1 bulk tag add/remove/update; canonify on note save 
also remove the tag list updated hook - we'll need a better solution in
the future than having the library code call back into the GUI code
 2020-05-12 21:13:34 +10:00
Damien Elmes
389b8a0536 add the Extra field back to cloze deletions as "Back Extra" 
Originally removed because some users were adding cloze deletions to
it, but removing it just replaced that problem with a new problem where
users add the field but don't add it to their card templates.
 2020-05-12 21:13:34 +10:00
Damien Elmes
7bab99d873 support disabling unicode normalization in notes 2020-05-12 21:13:34 +10:00
Damien Elmes
a7a485d550 use backend for genCards() and updateFieldCache() 2020-05-12 21:13:34 +10:00
Damien Elmes
1852e32183 update find&replace in browser 
sadly the UI still stutters on large selections - the calls
to get the selected rows from Qt are really slow.
 2020-05-12 21:13:34 +10:00
Damien Elmes
8b557ec382 move find&replace to backend 2020-05-12 21:13:34 +10:00
Damien Elmes
51bdbdb414 properly handle negated conditionals outside of req generation 2020-05-12 21:13:34 +10:00
Damien Elmes
2308b136fd add an index on notes(mid) so we can avoid the tablescan 2020-05-12 21:13:34 +10:00
Damien Elmes
996d408695 fix sort field check on save 2020-05-12 21:13:33 +10:00
Damien Elmes
5ff83fad08 catch attempts to overwrite notetype with older version 2020-05-12 21:13:33 +10:00
Damien Elmes
238441f2d9 use the backend for the deck due tree 
- approx 3x faster on a large test deck
- counts are no longer capped to 1000 in the tree
 2020-05-12 21:13:33 +10:00
Damien Elmes
5fb5338d97 add missing decks in backend 
- need to compare parents with unicode case folding
- duplicate check enforced by the DB
 2020-05-12 21:13:33 +10:00
Damien Elmes
67421e02ec add browserCollapsed as part of tree build 2020-05-12 21:13:33 +10:00
Damien Elmes
a88bc1e836 speed up browser load by rendering deck tree in Rust and skipping counts 2020-05-12 21:13:33 +10:00
Damien Elmes
5cefece264 switch primary key to deck id 2020-05-12 21:13:33 +10:00
Damien Elmes
8d17edc1e6 rework aux table sorting 
Instead of inserting text into the aux table and sorting on the fly
(which does a binary sort), we insert the ids in unicase-sorted order,
and then sort based on the automatically assigned rowids - giving
us faster sorts, and proper unicase folding.
 2020-05-12 21:13:33 +10:00
Damien Elmes
8246ba148e fix case sensitivity of notetype/template searches 2020-05-12 21:13:33 +10:00
Damien Elmes
3339c404b4 update searching code to use decks table 2020-05-12 21:13:33 +10:00
Damien Elmes
3ffb37270d add partial index on odid, and search odid as well 
the odid != 0 check in cards_for_deck.sql is necessary for sqlite
to be able to take advantage of the index
 2020-05-12 21:13:33 +10:00
Damien Elmes
f592672fa9 add separate decks table, and start on moving deck handling to Rust 
The Python tests are passing, but there are still a number of issues
to work through, and the table/protobuf schema is not yet finalized.
 2020-05-12 21:13:33 +10:00
Damien Elmes
36531ea96d add (currently disabled) check for missing/empty decks on card add 2020-05-12 21:13:33 +10:00
Damien Elmes
d3b27c302c split decks into module 2020-05-12 21:13:33 +10:00
Damien Elmes
bb56e9bc20 remove notetype from cache when removing 2020-05-12 21:13:33 +10:00
Damien Elmes
6e1d2990a0 remove unused _updateRequired and associated code 2020-05-12 21:13:33 +10:00
Damien Elmes
8bde0d4ac1 remove :{} and leading/trailing whitespace from field names on save 2020-05-12 21:13:33 +10:00
Damien Elmes
6e8860cafa hook the empty cards code up to the GUI 2020-05-12 21:13:33 +10:00
Damien Elmes
8b0121b0ac sort empty cards by notetype name 2020-05-12 21:13:33 +10:00
Damien Elmes
b476d071e7 don't include non-empty notes in the list 2020-05-12 21:13:33 +10:00
Damien Elmes
5ac3fb5514 handle negative due numbers when gathering existing cards 2020-05-12 21:13:33 +10:00
Damien Elmes
bee0eb1264 empty card handling 2020-05-12 21:13:33 +10:00
Damien Elmes
87cd119216 fix use counts not including unused note types 2020-05-12 21:13:33 +10:00
Damien Elmes
a181d9aa02 note type removal 2020-05-12 21:13:33 +10:00
Damien Elmes
d32935382d update template on field removals as well 2020-05-12 21:13:33 +10:00
Damien Elmes
d6706e1f0e handle changed sort field index 2020-05-12 21:13:33 +10:00
Damien Elmes
ea8e0ef6a2 update template when fields renamed 2020-05-12 21:13:33 +10:00
Damien Elmes
fb578a0c2d switch to owned strings in ParsedTemplate 
will make it easier to cache the parsed results in the future,
and handle field renames & other transformations
 2020-05-12 21:13:33 +10:00
Damien Elmes
ca5843acea only compare to previous note type if it exists 2020-05-12 21:13:33 +10:00
Damien Elmes
5471f5b1bf adjust sort index on save 2020-05-12 21:13:33 +10:00
Damien Elmes
6cc2bdbf87 start on exposing notes and individual note type methods 
changes to note:

- add_note() now takes a provided deck id instead of looking it up
in the notetype
- note type use counts fetched using a single table scan
- make sure note type changes are persisted
- expose optionalness of ords in templates and fields json
 2020-05-12 21:13:33 +10:00
Damien Elmes
b89dd32f78 mod schema if field/templates changed 
Instead of throwing an error if schema not marked as changed,
just mark it changed, as that way it can be included as part of
the same transaction.
 2020-05-12 21:13:33 +10:00
Damien Elmes
f86c2dc567 normal note types now generate a dummy card if required 
In the cloze deletion case, we already created a dummy card 0 when
no cloze deletions were found. This change makes normal note types
behave the same way - if no cards would be generated, a dummy card
0 is added to allow the note to be added.

This also applies when modifying note types - it is now possible
to delete card templates even if some notes only use that template,
as a dummy card 0 will be generated for notes that end up with no
cards left.
 2020-05-12 21:13:33 +10:00
Damien Elmes
5c3e5c672f use default deck for new note types 2020-05-12 21:13:33 +10:00
Damien Elmes
15ff279a96 cloze generation 2020-05-12 21:13:33 +10:00
Damien Elmes
83bcb084fe template changes and card generation 
Cloze cards are not yet supported, missing decks are not handled,
and more testing is still required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
c34e3e8096 fix infinite loop on unclosed {{/<% 2020-05-12 21:13:33 +10:00
Damien Elmes
a226f70733 cache notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
e8f4a75bd5 always fetch full notetype 2020-05-12 21:13:33 +10:00
Damien Elmes
527c8bf867 embed enum to avoid conflicts 2020-05-12 21:13:33 +10:00
Damien Elmes
f75fd5335d more work towards note adding 
Still a prototype at this stage - we'll likely want a caching layer
for note types, and I'm not sure of the merit of having fields in
a separate table, since they're almost always required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
5d4f9dc3c0 handle legacy syntax in parser instead of modifying original template 
Simplifies things for the caller, and ensures legacy handling doesn't
get accidentally forgotten
 2020-05-12 21:13:33 +10:00
Damien Elmes
f24dc05c8d the start of note adding, and note type changes 2020-05-12 21:13:33 +10:00
Damien Elmes
7c23deb562 add separate search_notes_only() 2020-05-12 21:13:33 +10:00
Damien Elmes
04f0ea8599 start work on field changes, and add search_cards to col 2020-05-12 21:13:33 +10:00
Damien Elmes
cc297f1dc6 move note db code to storage 2020-05-12 21:13:33 +10:00
Damien Elmes
da29d0fcae move latex header and styling into separate files 2020-05-12 21:13:33 +10:00
Damien Elmes
a17ddfdccd native struct for CardTemplate 2020-05-12 21:13:33 +10:00
Damien Elmes
9080f602b1 native struct for NoteField 
ord is made optional so we can keep track of changes in the future
 2020-05-12 21:13:33 +10:00
Damien Elmes
8cfb1fad44 use native struct for NoteType 2020-05-12 21:13:33 +10:00
Damien Elmes
d6646a6db5 expose stock notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
540892639f add stock note types in backend 2020-05-12 21:13:33 +10:00
Damien Elmes
7811a04df8 add default note type translations to ftl 2020-05-12 21:13:33 +10:00
Damien Elmes
32bc1e88eb migrate the schema11 usages to new structs/sql queries 2020-05-12 21:13:33 +10:00
Damien Elmes
e2b978e7cb move schema11 note type code into separate file 2020-05-12 21:13:33 +10:00
Damien Elmes
805a3a710e split note types into separate tables 
- store the config in protobuf instead of json
- still loading+saving in bulk for now
- code using the schema11 structs needs to be migrated
 2020-05-12 21:13:33 +10:00
Damien Elmes
66809dd8a3 ignore empty sound tags 
https://github.com/ankitects/anki/pull/612
 2020-05-12 20:53:50 +10:00
evandrocoan
b31bdd13a1 Also bump rslib version on make_check 
08ae4e7900
 2020-05-11 13:29:12 -03:00
Damien Elmes
08ae4e7900 update crate version 
make check was missing this
 2020-05-11 20:41:16 +10:00
Damien Elmes
1de8d4baaf fix a unit test 2020-05-08 17:27:41 +10:00
Damien Elmes
3f534b8c86 fix dupe search with included formatting 
https://anki.tenderapp.com/discussions/ankidesktop/41286-2125-doesnt-show-duplicates
 2020-05-08 15:26:45 +10:00
evandrocoan
2008f2b7bc Fixed rspy check not being called 
# Conflicts:
#	Makefile
 2020-05-01 22:51:17 -03:00
Damien Elmes
36ea980725 update rslib version 2020-05-02 10:46:58 +10:00
Damien Elmes
38a109a5bf tolerate decks with missing mtime exported by third party software 2020-05-02 10:18:08 +10:00
Damien Elmes
d7cdb10aa8 fix some clippy lints 2020-05-02 08:55:27 +10:00
Damien Elmes
898801ebda reset due on load if outside i32 bounds 
Some third party clients that create apkg files appear to be saving
a millisecond timestamp into this column. While this could also be
solved by increasing this field to a 64 bit number, some older clients
expect this column to be a 32 bit number.
 2020-04-30 11:57:35 +10:00
Damien Elmes
c95983ac1f preserve entities when stripping HTML for MathJax 
https://anki.tenderapp.com/discussions/ankidesktop/40987-how-to-render-angled-brackets
 2020-04-30 11:17:38 +10:00
Damien Elmes
e27d015ff9 expose clock hack via env var 
closes #594
 2020-04-30 09:33:02 +10:00
Damien Elmes
a4d33dbdb0 report filename when unable to read data 2020-04-30 08:02:08 +10:00
Damien Elmes
18a6218a77 don't throw an error when a file is greater than MAXPATH 
https://anki.tenderapp.com/discussions/ankidesktop/40931-check-media-error
 2020-04-30 08:01:45 +10:00
Damien Elmes
4e33775ed4 drop type answer requirement special case 
While it makes no sense to have the typing box on the front without
another non-empty field, "the front of this card is empty" when
a typing box is on the template seems to be confusing some users, so
it is clearer to show them the box.
 2020-04-29 21:29:16 +10:00
Damien Elmes
d36bd33972 AnkiDroid expects timer to be an integer 
https://github.com/ankidroid/Anki-Android/issues/6089#issuecomment-621110598
 2020-04-29 20:52:00 +10:00
evandrocoan
f6f1e3cbb3 Added an warning that rslib/build.rs is automatically generated file 2020-04-29 01:21:49 -03:00
Damien Elmes
00cf4fc585 bump version 2020-04-28 08:57:56 +10:00
evandrocoan
26480fb13c Fixed make clean rule not working on Windows 2020-04-26 03:02:02 -03:00
Damien Elmes
8dfd362fed support globbing chars inside word boundary search 2020-04-25 09:43:08 +10:00
Damien Elmes
fd844a0d5a add w: shortcut for searching on word boundaries 2020-04-25 08:48:01 +10:00
Damien Elmes
37bc44a8f5 report the missing filename in media check 
https://anki.tenderapp.com/discussions/ankidesktop/40578-check-media-error
 2020-04-17 17:17:42 +10:00
Damien Elmes
f18f4e387b older clients require 'separate' on filtered decks 
the deckconf['new']['separate' property is unused, but older code
assumes filtered['separate'] exists, even though it doesn't do anything
with it

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48240796
 2020-04-15 15:02:12 +10:00
Damien Elmes
b55742509a embed sort kind enum to avoid future namespace conflicts 2020-04-14 20:31:10 +10:00
Damien Elmes
6e6776aab2 fix dyn being serialized as bool 
If you were running from git in the last ~30 hours, please load
your collection using the latest git, review one card, then force
a full upload to fix compatibility with other clients.
 2020-04-10 18:07:59 +10:00
Damien Elmes
bbceeacf0b merge the 12-14 upgrade code 2020-04-10 17:46:57 +10:00
Damien Elmes
11fdd96a44 add socks support 2020-04-09 14:28:36 +10:00
Damien Elmes
92fbe3bfba update rusqlite 2020-04-09 12:45:51 +10:00
Damien Elmes
4c7210b430 (de)serialize decks in backend 2020-04-09 12:41:59 +10:00
Damien Elmes
36ec7830a9 load/save note types in backend 
This allows us to normalize bad data, and is the first step towards
splitting note types into separate tables.
 2020-04-08 10:05:07 +10:00
Damien Elmes
c29184d874 skip terminal logging unless LOGTERM defined 
seems to be quite slow on Windows
 2020-04-07 10:25:47 +10:00
Damien Elmes
c950a00610 Cow<str> workaround no longer required 2020-04-07 10:08:03 +10:00
Damien Elmes
caa513899d update rusqlite for param count assert fix 2020-04-07 08:54:57 +10:00
Damien Elmes
62fa265213 avoid warning on int sortBackwards from older Anki versions 2020-04-06 17:11:03 +10:00
Damien Elmes
75faba1667 add separate json error 2020-04-06 16:29:31 +10:00
Damien Elmes
03db16460a log to the terminal as well 2020-04-06 15:57:13 +10:00
Damien Elmes
dc62d5072c use bytes for deck config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
164bd29438 use bytes for config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
5a0203c7b4 display warning when config load fails 2020-04-06 15:39:47 +10:00
Damien Elmes
34cca119e3 use blobs for config 2020-04-06 15:39:47 +10:00
Damien Elmes
676f4e74a8 store config in separate DB table 
- mtime is tracked on each key individually, which will allow
merging of config changes when syncing in the future
- added col.(get|set|remove)_config()
- in order to support existing code that was mutating returned
values (eg col.conf["something"]["another"] = 5), the returned list/dict
will be automatically wrapped so that when the value is dropped, it
will save the mutated item back to the DB if it's changed. Code that
is fetching lists/dicts from the config like so:

col.conf["foo"]["bar"] = baz
col.setMod()

will continue to work in most case, but should be gradually updated to:

conf = col.get_config("foo")
conf["bar"] = baz
col.set_config("foo", conf)
 2020-04-06 15:39:47 +10:00
Damien Elmes
0750ad5c62 fix duplicates search 2020-04-04 17:38:35 +10:00
Damien Elmes
7375a0389a add flag to skip downgrade on collection close 
Disabled for now; when enabled it will allow faster collection
open and close in the normal case, while continuing to downgrade
when exporting or doing a full sync.

Also, when downgrading is disabled, the journal mode is no longer
changed back to delete.
 2020-04-04 17:21:45 +10:00
Damien Elmes
45ea0ddefd use separate schema version for tag changes 
Will prevent issues if user upgrades after an unclean shutdown
 2020-04-04 10:14:51 +10:00
Damien Elmes
3b619c4ca5 use case folding when sorting in canonify 2020-04-03 19:34:46 +10:00
Damien Elmes
ac4284b2de update tag handling 
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
 2020-04-03 19:34:46 +10:00
Damien Elmes
333d0735ff preserve mtime/usn when syncing deck config, and add snake_case names 2020-04-03 19:34:46 +10:00
Damien Elmes
ee71e5dd59 use support URL and point to source repo 2020-04-03 08:44:59 +10:00
evandrocoan
069cc11a86 Created the GitHub Actions step Upload to PyPi to run only on 
tagged releases of anki:
https://anki.tenderapp.com/discussions/ankidesktop/40025-anki-in-python-package-index
 2020-04-02 02:57:34 -03:00
Damien Elmes
70a1992f70 fix is:due search 2020-04-02 09:33:05 +10:00
Damien Elmes
c68a2fc838 return interrupted err when media sync aborted 2020-04-01 21:40:14 +10:00
Damien Elmes
05cc47a5d3 fix push-i18n-ftl 2020-04-01 18:20:31 +10:00
Damien Elmes
546f4b44fc FString -> TR 2020-04-01 18:16:43 +10:00
Damien Elmes
dfa1ce6429 translate default deck config name 2020-04-01 17:49:25 +10:00
Damien Elmes
a5940e3557 add fixme 2020-04-01 17:36:33 +10:00
Damien Elmes
d396460332 ignore .DS_Store in trash 
https://anki.tenderapp.com/discussions/ankidesktop/40035-anki-2123-on-macos-ds_store-files-in-new-anki-trash
 2020-04-01 13:37:57 +10:00
Damien Elmes
6495240914 add some timeouts 
I'm waiting to hear back from the reqwests author on the best way to
handle this. For now, this change just adds short timeouts
to the quick requests, and will wait for up to an hour when sending/
receiving files to allow for the slow connection+large file case.
 2020-04-01 10:03:54 +10:00
Damien Elmes
c9da4bc1a6 allow aborting a media sync while http req in flight 
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
 2020-04-01 09:49:25 +10:00
Damien Elmes
35c03af520 split deck config into separate SQL table 
- on collection load, the schema is upgraded to 12
- on collection close, the changes are reversed so older clients
can continue to open the collection
- in the future, we could potentially skip the reversal except
when exporting/doing a full sync
- the same approach should work for decks, note types and tags in the
future too
- the deck list code needs updating to cache the deck confs for the
life of the call
 2020-03-30 20:01:16 +10:00
Damien Elmes
004cc2b5f8 move deck conf handling to backend 2020-03-30 14:39:46 +10:00
Damien Elmes
bf83715ee0 initial work on undo support 2020-03-29 17:52:16 +10:00
Damien Elmes
f90e5dbe2c remove usn cache 
SqliteStorage no longer contains any mutable state
 2020-03-29 12:50:38 +10:00
Damien Elmes
d1ecf33c72 cache timing_today in collection, update it when cutover reached 2020-03-29 12:26:24 +10:00
Damien Elmes
fdeca610b0 drop separate RequestContext/StorageContext 2020-03-29 12:12:35 +10:00
Damien Elmes
2810d3883b roll back the vec cache changes 
prepare_cached() is sufficiently fast, and allowing the vec cache
to persist across calls is complicated due to lifetime restrictions
 2020-03-29 09:26:09 +10:00
Damien Elmes
1a1a00d50f Revert "add benchmark for vec cache" 
This reverts commit 82ed288dc5.
 2020-03-29 09:16:11 +10:00
Damien Elmes
f6219b59ac
Merge pull request #528 from evandroforks/create_actions_for_windows_macos 
Create actions for Windows and Mac OS
 2020-03-27 15:26:22 +10:00
Damien Elmes
84b84ae31c use add_card() 2020-03-27 15:11:07 +10:00
Damien Elmes
6b9378fb41 add (unused) add_card, and move sql to separate files 2020-03-27 14:48:25 +10:00
Damien Elmes
82ed288dc5 add benchmark for vec cache 
test storage::sqlite::bench::bench_hash_cache ... bench:         399 ns/iter (+/- 27)
test storage::sqlite::bench::bench_no_cache   ... bench:       4,854 ns/iter (+/- 499)
test storage::sqlite::bench::bench_vec_cache  ... bench:           0 ns/iter (+/- 0)
 2020-03-27 09:59:48 +10:00
Damien Elmes
2f4e35d566 store cached statements in a vec instead of separate optionals 2020-03-27 09:51:19 +10:00
evandrocoan
b07454ca0c Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 2020-03-26 19:31:01 -03:00
Damien Elmes
369d2d89d9 update card in transaction 2020-03-27 07:58:25 +10:00
Damien Elmes
7f16f436c3 handle mtime/usn bump in backend, and tweak integer sizes 2020-03-26 20:55:04 +10:00
Damien Elmes
7a56d18539 report invalid type/queue 2020-03-26 19:32:35 +10:00
Damien Elmes
47504245dc update_card() 2020-03-26 18:54:20 +10:00
Damien Elmes
7ddaf93f7a use backend to get card 2020-03-26 17:47:43 +10:00
Damien Elmes
8abba00496 DeckID, CardID 2020-03-26 15:10:40 +10:00
Damien Elmes
f52e775354 NoteTypeID 2020-03-26 15:00:24 +10:00
Damien Elmes
4ec30e412a newtype NoteID 2020-03-26 14:42:43 +10:00
Damien Elmes
5eed3d7f71 use a macro for newtype defs 2020-03-26 13:50:20 +10:00
Damien Elmes
39f916a23e usn newtype 2020-03-26 13:06:02 +10:00
Damien Elmes
eb89a2db3f use newtypes for distinguishing between second and millisecond stamps 2020-03-26 12:59:51 +10:00
evandrocoan
721453f923 Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	.github/workflows/checks.yml
#	Makefile
#	pylib/Makefile
#	qt/Makefile
#	qt/tools/typecheck-setup.sh
#	rspy/Makefile
 2020-03-24 22:49:49 -03:00
Damien Elmes
01e9f889b7 add newline to mear clearly separate different ftl files 2020-03-25 08:53:42 +10:00
Damien Elmes
289bdde20c handle fields with embedded nuls 2020-03-24 14:53:33 +10:00
evandrocoan
20c9837770 Created the .github/workflows/windows.yml 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	Makefile
#	qt/tools/typecheck-setup.sh
#	react/Makefile
#	rspy/Makefile
#	svelte/Makefile
#	tslib/Makefile
 2020-03-23 19:51:03 -03:00
Damien Elmes
11a4d582b4 convert asc to desc instead of appending desc to the end of the order 
as the latter doesn't work when sorting on more than one column

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48174812
 2020-03-23 19:53:57 +10:00
Damien Elmes
14a970e923 rename long filenames in fields if files renamed in a previous sync 2020-03-23 18:38:45 +10:00
Damien Elmes
4e2e0d1b84 fix setting of wal 2020-03-23 13:52:57 +10:00
Damien Elmes
430f1ad616 handle trailing whitespace inside group 2020-03-22 18:52:44 +10:00
Damien Elmes
25ff4642ec accept now_mins_west for v2 legacy timing as well 2020-03-22 17:33:14 +10:00
Damien Elmes
69d8cdd9ed use backend for v1 and v2 cutoff calculations 
this should also fix the CI failures, which were happening because
the datetime module wasn't matching the shifted time.time()
 2020-03-22 14:28:26 +10:00
Damien Elmes
c5629e96df exclude autogenerated src from build deps 
prevents unnecessary rebuilds
 2020-03-22 03:39:38 +00:00
Damien Elmes
47fcdd0723 possible fix for CI failure 2020-03-22 13:17:00 +10:00
Damien Elmes
f28e57a367 add enum for controlling sort order 
eg col.find_cards("", order=BuiltinSortKind.CARD_DUE)
 2020-03-22 12:59:24 +10:00
Damien Elmes
199713a39a handle collections with sortBackwards set to 0 instead of a bool 2020-03-22 10:26:09 +10:00
Damien Elmes
cc44523449 remove debugging line 2020-03-21 18:29:04 +10:00
Damien Elmes
9696e959be increase the rated search cap to 365, and allow searches for ease 0 
An add-on appears to use ease 0 when rescheduling cards, and it may
make sense for Anki to do the same in the future as well.
 2020-03-21 16:05:17 +10:00
Damien Elmes
51a379de23 add search that ignores combining chars 
On a test of a ~40k card collection, the 'ignore accents' add-on
takes about 1150ms, and this code takes about 70ms.
 2020-03-21 15:15:59 +10:00
Damien Elmes
f0ed34d79b support regexp search in single field 2020-03-21 13:06:46 +10:00
Damien Elmes
97577dbc16 support wildcard in field*:val search 2020-03-21 12:45:25 +10:00
Damien Elmes
08e64d246d don't require wildcard for unicode case folding in search 2020-03-21 12:44:56 +10:00
Damien Elmes
4ff17d31b3 add unicase collation 
sqlite's like is hard-coded to use ASCII comparisons, so we can't
take advantage of this yet
 2020-03-21 12:40:20 +10:00
Damien Elmes
d1ebdbdcce support regex searches 2020-03-21 12:00:48 +10:00
Damien Elmes
2dc1b5c982 add regexp() to sqlite 2020-03-21 10:23:11 +10:00
Damien Elmes
2aab44d9ce support deck:"foo bar" style searches 2020-03-21 09:34:24 +10:00
Damien Elmes
a5787781d7 add note searching 2020-03-21 09:00:05 +10:00
Damien Elmes
aee64016ac fix formatting and unit test 2020-03-21 08:17:24 +10:00
Damien Elmes
3a4146560c handle escaped tag searches and tag:* special case 2020-03-21 08:09:28 +10:00
Damien Elmes
ad09c89c3c check for child decks case-insensitively 2020-03-21 07:57:07 +10:00
Damien Elmes
c3314d3689 don't crash when card:0 passed in 2020-03-21 07:56:55 +10:00
Damien Elmes
124357bd82 handle * in single-field search 2020-03-21 07:56:32 +10:00
Damien Elmes
949252d438 fix ease search 2020-03-21 07:56:09 +10:00
Damien Elmes
5debd3e0f8 add the ability to provide a custom sort order; use backend for find 2020-03-21 07:55:21 +10:00
Damien Elmes
09a76967e7 support sorting on tags 
I don't personally understand it, but some users seem to want it.
 2020-03-20 21:15:23 +10:00
Damien Elmes
00d0447ecb support sorting on note type, card template and decks 2020-03-20 21:15:23 +10:00
Damien Elmes
2c362d6991 search order 2020-03-20 21:15:23 +10:00
Damien Elmes
d94effcdc7 fix is:new/is:review 2020-03-20 21:15:23 +10:00
Damien Elmes
79697746a4 added needs to use milliseconds 2020-03-20 21:15:23 +10:00
Damien Elmes
c90670ec3a tolerate some string IDs 2020-03-20 21:15:23 +10:00
Damien Elmes
224bad2566 handle empty searches and leading/trailing whitespace 2020-03-20 21:15:23 +10:00
Damien Elmes
67cb27bada add remaining tests and fix some clippy lints 2020-03-20 21:15:23 +10:00
Damien Elmes
8c158a3897 fix rated and state searches 2020-03-20 21:15:23 +10:00
Damien Elmes
37ad664afc fix tags, more tests 2020-03-20 21:15:23 +10:00
Damien Elmes
fa654a0e22 fix cards 2020-03-20 21:15:23 +10:00
Damien Elmes
425a9d04ca fix decks 2020-03-20 21:15:23 +10:00
Damien Elmes
2693e142aa fix added 2020-03-20 21:15:23 +10:00
Damien Elmes
5df04b161c fix qualified search 2020-03-20 21:15:23 +10:00
Damien Elmes
c723adea17 fix escape handling, and handle sql wildcards 2020-03-20 21:15:23 +10:00
Damien Elmes
b70668d31c avoid extra sql binding in unqualified search 2020-03-20 21:15:23 +10:00
Damien Elmes
1f9e8e388a start on search tests 2020-03-20 21:15:23 +10:00
Damien Elmes
bca5f2ddff prototype of integration 
no ordering yet, and no tests
 2020-03-20 21:15:23 +10:00
Damien Elmes
9752de5aaa finish the remaining searches 
Searches that require multiple deck or note type lookups won't perform
very well at the moment - it either needs caching or to be split up
at the DB level.

Nothing tested yet.
 2020-03-20 21:15:23 +10:00
Damien Elmes
85af35509d ctx->req 2020-03-20 21:15:23 +10:00
Damien Elmes
dc12c23ce9 add timing to search 2020-03-20 21:15:23 +10:00
Damien Elmes
2beccd377b add v1 and v2 legacy timing code 2020-03-20 21:15:23 +10:00
Damien Elmes
9f3cc0982d deck searching 
A bit more complicated than it needs to be, as we don't have the
full deck manager infrastructure yet.
 2020-03-20 21:15:23 +10:00
Damien Elmes
f559ae3ef8 address some clippy lints 2020-03-20 21:15:23 +10:00
Damien Elmes
cc54e92756 move html stripping out of field_checksum into caller 2020-03-20 21:15:23 +10:00
Damien Elmes
7eab504126 add field_at_index() sql func 2020-03-20 21:15:23 +10:00
Damien Elmes
cffa52ff82 more searching work 2020-03-20 21:15:23 +10:00
Damien Elmes
761d1d1812 add card queue/type enums 2020-03-20 21:15:23 +10:00
Damien Elmes
91d7b02325 separate out template ordinal and name search 2020-03-20 21:15:23 +10:00
Damien Elmes
dcb2b46d1b use .eq_ignore_ascii_case() to avoid allocating 2020-03-20 21:15:23 +10:00
Damien Elmes
68657c7166 field_checksum needs to strip HTML 2020-03-20 21:15:23 +10:00
Damien Elmes
4f93ae4b6d start of searching sql 2020-03-20 21:15:23 +10:00
Damien Elmes
00300bb24d ensure rated ease in range 2020-03-20 21:15:23 +10:00
Damien Elmes
e790367b1e ensure id list not empty 2020-03-20 21:15:23 +10:00
Damien Elmes
08d205d377 decode search terms in parser 2020-03-20 21:15:23 +10:00
Damien Elmes
289318d92c split up searches with a qualifier 2020-03-20 21:15:23 +10:00
Damien Elmes
f623f19b3d basic search parsing 2020-03-20 21:15:23 +10:00
Damien Elmes
794c8a984b add string about waiting for completion 2020-03-20 21:15:23 +10:00
Damien Elmes
72bcef917e release mutex before beginning media sync 
And check media sync is not running on close
 2020-03-20 21:15:23 +10:00
Damien Elmes
5f19048c93 fix media sync being logged to console 2020-03-20 21:15:23 +10:00
Damien Elmes
ba17567617 drop the separate i18n backend 2020-03-20 21:15:23 +10:00
Damien Elmes
874bc085fe support opening and closing the DB while keeping backend alive 
This is safer than just dropping the backend, as .close() will
block if something else is holding the mutex. Also means we can
drop the extra I18nBackend code.

Media syncing still needs fixing.
 2020-03-20 21:15:23 +10:00
Damien Elmes
94e4c40ebf move media folder/db paths into collection 
this breaks background media syncing for now
 2020-03-20 21:15:23 +10:00
Damien Elmes
649b40371b drop unused col_path 2020-03-20 21:15:23 +10:00
Damien Elmes
1322d8c617 make the collection optional 2020-03-20 21:15:23 +10:00
Damien Elmes
db1508e27c support first_row_only 2020-03-20 21:15:23 +10:00
Damien Elmes
e14c5e4745 proper implementation of executemany(); drop executescript() 2020-03-20 21:15:23 +10:00
Damien Elmes
14546c8a8b wrap the collection in a mutex so DB access is thread safe 2020-03-20 21:15:23 +10:00
Damien Elmes
47c142a74c move note code into notes.rs, add ability to rollback when unchanged 2020-03-20 21:15:23 +10:00
Damien Elmes
ae06b9e446 add Collection struct, and get media check working again 
- media check no longer needs collection to be closed
- use savepoints for operations initiated by Rust, so they are
atomic without forcing a commit
 2020-03-20 21:15:23 +10:00
Damien Elmes
2cd7885ec0 add begin/commit/rollback, and support creating collections 
all but one unit test is now passing
 2020-03-20 21:15:23 +10:00
Damien Elmes
b51e575a9d drop the protobuf prototype 2020-03-20 21:15:23 +10:00
Damien Elmes
b876d97770 use (or)json for DB bridge 
Some initial testing with orjson indicates performance varies from
slightly better than pysqlite to about 2x slower depending on the type
of query.

Performance could be improved by building the Python list in rspy
instead of sending back json that needs to be decoded, but it may make
more sense to rewrite the hotspots in Rust instead. More testing is
required in any case.
 2020-03-20 21:15:23 +10:00
Damien Elmes
04ca8ec038 initial work on DB handling in Rust 
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
 2020-03-20 21:15:23 +10:00
Damien Elmes
e3a57a4193 fix clippy lint on other platforms 2020-03-19 10:58:52 +10:00
Damien Elmes
f617760d04 bump version 2020-03-19 10:46:01 +10:00
Damien Elmes
617b18ff49 correctly handle NFD content in media DB from older Anki versions 2020-03-19 10:20:42 +10:00
Damien Elmes
d457ab0b17 add helper to duplicate a fluent string 2020-03-12 19:53:44 +10:00
Damien Elmes
f0be553e09 add media check window title, use consistent formatting in template 2020-03-12 18:02:22 +10:00
Damien Elmes
5e676db4a0 fix fallback preferences being used instead of English 
Check needs to be done regardless of if we found a regional variant
or not.
 2020-03-12 18:02:15 +10:00
Damien Elmes
9736e4a970 ignore remote file links in media check 2020-03-10 13:46:52 +10:00
Damien Elmes
0f4c3ab611 add restore media action 2020-03-10 13:35:09 +10:00
Damien Elmes
6ad2a1f9a3 add empty trash action in aqt 2020-03-10 12:49:40 +10:00
Damien Elmes
255d6f8af7 don't clean up the trash folder automatically 
The media check now reports if any files are in the trash folder,
and a method has been exposed to clear the trash.
 2020-03-10 12:02:38 +10:00
Damien Elmes
67d6b4f713 add basic log rotation 2020-03-10 10:33:14 +10:00
Damien Elmes
6e30572e6d log sync errors 2020-03-10 09:43:48 +10:00
Damien Elmes
b4af0ca825 tweak logging 2020-03-10 09:30:05 +10:00
Damien Elmes
18cd841246 don't try to read file data when file marked as a deletion 
If the user hadn't used the media check to clean up files with
invalid names on disk,  the long filename would break the 'automatically
rename long filenames when syncing' functionality, causing the original
long filename to be uploaded to AnkiWeb as an addition instead of
a deletion
 2020-03-10 09:29:36 +10:00
Damien Elmes
93bb5a0bf6 fix missing newline in ftl files 
https://anki.tenderapp.com/discussions/beta-testing/1836-problem-with-new-translations
 2020-03-09 18:58:41 +10:00