langfingaz/anki
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,461 Commits 2 Branches 2 Tags 39 MiB
c77d017135
Commit Graph

525 Commits

Author SHA1 Message Date
Damien Elmes
a88bc1e836 speed up browser load by rendering deck tree in Rust and skipping counts 2020-05-12 21:13:33 +10:00
Damien Elmes
5cefece264 switch primary key to deck id 2020-05-12 21:13:33 +10:00
Damien Elmes
8d17edc1e6 rework aux table sorting 
Instead of inserting text into the aux table and sorting on the fly
(which does a binary sort), we insert the ids in unicase-sorted order,
and then sort based on the automatically assigned rowids - giving
us faster sorts, and proper unicase folding.
 2020-05-12 21:13:33 +10:00
Damien Elmes
8246ba148e fix case sensitivity of notetype/template searches 2020-05-12 21:13:33 +10:00
Damien Elmes
3339c404b4 update searching code to use decks table 2020-05-12 21:13:33 +10:00
Damien Elmes
3ffb37270d add partial index on odid, and search odid as well 
the odid != 0 check in cards_for_deck.sql is necessary for sqlite
to be able to take advantage of the index
 2020-05-12 21:13:33 +10:00
Damien Elmes
f592672fa9 add separate decks table, and start on moving deck handling to Rust 
The Python tests are passing, but there are still a number of issues
to work through, and the table/protobuf schema is not yet finalized.
 2020-05-12 21:13:33 +10:00
Damien Elmes
36531ea96d add (currently disabled) check for missing/empty decks on card add 2020-05-12 21:13:33 +10:00
Damien Elmes
d3b27c302c split decks into module 2020-05-12 21:13:33 +10:00
Damien Elmes
bb56e9bc20 remove notetype from cache when removing 2020-05-12 21:13:33 +10:00
Damien Elmes
6e1d2990a0 remove unused _updateRequired and associated code 2020-05-12 21:13:33 +10:00
Damien Elmes
8bde0d4ac1 remove :{} and leading/trailing whitespace from field names on save 2020-05-12 21:13:33 +10:00
Damien Elmes
6e8860cafa hook the empty cards code up to the GUI 2020-05-12 21:13:33 +10:00
Damien Elmes
8b0121b0ac sort empty cards by notetype name 2020-05-12 21:13:33 +10:00
Damien Elmes
b476d071e7 don't include non-empty notes in the list 2020-05-12 21:13:33 +10:00
Damien Elmes
5ac3fb5514 handle negative due numbers when gathering existing cards 2020-05-12 21:13:33 +10:00
Damien Elmes
bee0eb1264 empty card handling 2020-05-12 21:13:33 +10:00
Damien Elmes
87cd119216 fix use counts not including unused note types 2020-05-12 21:13:33 +10:00
Damien Elmes
a181d9aa02 note type removal 2020-05-12 21:13:33 +10:00
Damien Elmes
d32935382d update template on field removals as well 2020-05-12 21:13:33 +10:00
Damien Elmes
d6706e1f0e handle changed sort field index 2020-05-12 21:13:33 +10:00
Damien Elmes
ea8e0ef6a2 update template when fields renamed 2020-05-12 21:13:33 +10:00
Damien Elmes
fb578a0c2d switch to owned strings in ParsedTemplate 
will make it easier to cache the parsed results in the future,
and handle field renames & other transformations
 2020-05-12 21:13:33 +10:00
Damien Elmes
ca5843acea only compare to previous note type if it exists 2020-05-12 21:13:33 +10:00
Damien Elmes
5471f5b1bf adjust sort index on save 2020-05-12 21:13:33 +10:00
Damien Elmes
6cc2bdbf87 start on exposing notes and individual note type methods 
changes to note:

- add_note() now takes a provided deck id instead of looking it up
in the notetype
- note type use counts fetched using a single table scan
- make sure note type changes are persisted
- expose optionalness of ords in templates and fields json
 2020-05-12 21:13:33 +10:00
Damien Elmes
b89dd32f78 mod schema if field/templates changed 
Instead of throwing an error if schema not marked as changed,
just mark it changed, as that way it can be included as part of
the same transaction.
 2020-05-12 21:13:33 +10:00
Damien Elmes
f86c2dc567 normal note types now generate a dummy card if required 
In the cloze deletion case, we already created a dummy card 0 when
no cloze deletions were found. This change makes normal note types
behave the same way - if no cards would be generated, a dummy card
0 is added to allow the note to be added.

This also applies when modifying note types - it is now possible
to delete card templates even if some notes only use that template,
as a dummy card 0 will be generated for notes that end up with no
cards left.
 2020-05-12 21:13:33 +10:00
Damien Elmes
5c3e5c672f use default deck for new note types 2020-05-12 21:13:33 +10:00
Damien Elmes
15ff279a96 cloze generation 2020-05-12 21:13:33 +10:00
Damien Elmes
83bcb084fe template changes and card generation 
Cloze cards are not yet supported, missing decks are not handled,
and more testing is still required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
c34e3e8096 fix infinite loop on unclosed {{/<% 2020-05-12 21:13:33 +10:00
Damien Elmes
a226f70733 cache notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
e8f4a75bd5 always fetch full notetype 2020-05-12 21:13:33 +10:00
Damien Elmes
527c8bf867 embed enum to avoid conflicts 2020-05-12 21:13:33 +10:00
Damien Elmes
f75fd5335d more work towards note adding 
Still a prototype at this stage - we'll likely want a caching layer
for note types, and I'm not sure of the merit of having fields in
a separate table, since they're almost always required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
5d4f9dc3c0 handle legacy syntax in parser instead of modifying original template 
Simplifies things for the caller, and ensures legacy handling doesn't
get accidentally forgotten
 2020-05-12 21:13:33 +10:00
Damien Elmes
f24dc05c8d the start of note adding, and note type changes 2020-05-12 21:13:33 +10:00
Damien Elmes
7c23deb562 add separate search_notes_only() 2020-05-12 21:13:33 +10:00
Damien Elmes
04f0ea8599 start work on field changes, and add search_cards to col 2020-05-12 21:13:33 +10:00
Damien Elmes
cc297f1dc6 move note db code to storage 2020-05-12 21:13:33 +10:00
Damien Elmes
da29d0fcae move latex header and styling into separate files 2020-05-12 21:13:33 +10:00
Damien Elmes
a17ddfdccd native struct for CardTemplate 2020-05-12 21:13:33 +10:00
Damien Elmes
9080f602b1 native struct for NoteField 
ord is made optional so we can keep track of changes in the future
 2020-05-12 21:13:33 +10:00
Damien Elmes
8cfb1fad44 use native struct for NoteType 2020-05-12 21:13:33 +10:00
Damien Elmes
d6646a6db5 expose stock notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
540892639f add stock note types in backend 2020-05-12 21:13:33 +10:00
Damien Elmes
7811a04df8 add default note type translations to ftl 2020-05-12 21:13:33 +10:00
Damien Elmes
32bc1e88eb migrate the schema11 usages to new structs/sql queries 2020-05-12 21:13:33 +10:00
Damien Elmes
e2b978e7cb move schema11 note type code into separate file 2020-05-12 21:13:33 +10:00
Damien Elmes
805a3a710e split note types into separate tables 
- store the config in protobuf instead of json
- still loading+saving in bulk for now
- code using the schema11 structs needs to be migrated
 2020-05-12 21:13:33 +10:00
Damien Elmes
66809dd8a3 ignore empty sound tags 
https://github.com/ankitects/anki/pull/612
 2020-05-12 20:53:50 +10:00
evandrocoan
b31bdd13a1 Also bump rslib version on make_check 
08ae4e7900
 2020-05-11 13:29:12 -03:00
Damien Elmes
08ae4e7900 update crate version 
make check was missing this
 2020-05-11 20:41:16 +10:00
Damien Elmes
1de8d4baaf fix a unit test 2020-05-08 17:27:41 +10:00
Damien Elmes
3f534b8c86 fix dupe search with included formatting 
https://anki.tenderapp.com/discussions/ankidesktop/41286-2125-doesnt-show-duplicates
 2020-05-08 15:26:45 +10:00
evandrocoan
2008f2b7bc Fixed rspy check not being called 
# Conflicts:
#	Makefile
 2020-05-01 22:51:17 -03:00
Damien Elmes
36ea980725 update rslib version 2020-05-02 10:46:58 +10:00
Damien Elmes
38a109a5bf tolerate decks with missing mtime exported by third party software 2020-05-02 10:18:08 +10:00
Damien Elmes
d7cdb10aa8 fix some clippy lints 2020-05-02 08:55:27 +10:00
Damien Elmes
898801ebda reset due on load if outside i32 bounds 
Some third party clients that create apkg files appear to be saving
a millisecond timestamp into this column. While this could also be
solved by increasing this field to a 64 bit number, some older clients
expect this column to be a 32 bit number.
 2020-04-30 11:57:35 +10:00
Damien Elmes
c95983ac1f preserve entities when stripping HTML for MathJax 
https://anki.tenderapp.com/discussions/ankidesktop/40987-how-to-render-angled-brackets
 2020-04-30 11:17:38 +10:00
Damien Elmes
e27d015ff9 expose clock hack via env var 
closes #594
 2020-04-30 09:33:02 +10:00
Damien Elmes
a4d33dbdb0 report filename when unable to read data 2020-04-30 08:02:08 +10:00
Damien Elmes
18a6218a77 don't throw an error when a file is greater than MAXPATH 
https://anki.tenderapp.com/discussions/ankidesktop/40931-check-media-error
 2020-04-30 08:01:45 +10:00
Damien Elmes
4e33775ed4 drop type answer requirement special case 
While it makes no sense to have the typing box on the front without
another non-empty field, "the front of this card is empty" when
a typing box is on the template seems to be confusing some users, so
it is clearer to show them the box.
 2020-04-29 21:29:16 +10:00
Damien Elmes
d36bd33972 AnkiDroid expects timer to be an integer 
https://github.com/ankidroid/Anki-Android/issues/6089#issuecomment-621110598
 2020-04-29 20:52:00 +10:00
evandrocoan
f6f1e3cbb3 Added an warning that rslib/build.rs is automatically generated file 2020-04-29 01:21:49 -03:00
Damien Elmes
00cf4fc585 bump version 2020-04-28 08:57:56 +10:00
evandrocoan
26480fb13c Fixed make clean rule not working on Windows 2020-04-26 03:02:02 -03:00
Damien Elmes
8dfd362fed support globbing chars inside word boundary search 2020-04-25 09:43:08 +10:00
Damien Elmes
fd844a0d5a add w: shortcut for searching on word boundaries 2020-04-25 08:48:01 +10:00
Damien Elmes
37bc44a8f5 report the missing filename in media check 
https://anki.tenderapp.com/discussions/ankidesktop/40578-check-media-error
 2020-04-17 17:17:42 +10:00
Damien Elmes
f18f4e387b older clients require 'separate' on filtered decks 
the deckconf['new']['separate' property is unused, but older code
assumes filtered['separate'] exists, even though it doesn't do anything
with it

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48240796
 2020-04-15 15:02:12 +10:00
Damien Elmes
b55742509a embed sort kind enum to avoid future namespace conflicts 2020-04-14 20:31:10 +10:00
Damien Elmes
6e6776aab2 fix dyn being serialized as bool 
If you were running from git in the last ~30 hours, please load
your collection using the latest git, review one card, then force
a full upload to fix compatibility with other clients.
 2020-04-10 18:07:59 +10:00
Damien Elmes
bbceeacf0b merge the 12-14 upgrade code 2020-04-10 17:46:57 +10:00
Damien Elmes
11fdd96a44 add socks support 2020-04-09 14:28:36 +10:00
Damien Elmes
92fbe3bfba update rusqlite 2020-04-09 12:45:51 +10:00
Damien Elmes
4c7210b430 (de)serialize decks in backend 2020-04-09 12:41:59 +10:00
Damien Elmes
36ec7830a9 load/save note types in backend 
This allows us to normalize bad data, and is the first step towards
splitting note types into separate tables.
 2020-04-08 10:05:07 +10:00
Damien Elmes
c29184d874 skip terminal logging unless LOGTERM defined 
seems to be quite slow on Windows
 2020-04-07 10:25:47 +10:00
Damien Elmes
c950a00610 Cow<str> workaround no longer required 2020-04-07 10:08:03 +10:00
Damien Elmes
caa513899d update rusqlite for param count assert fix 2020-04-07 08:54:57 +10:00
Damien Elmes
62fa265213 avoid warning on int sortBackwards from older Anki versions 2020-04-06 17:11:03 +10:00
Damien Elmes
75faba1667 add separate json error 2020-04-06 16:29:31 +10:00
Damien Elmes
03db16460a log to the terminal as well 2020-04-06 15:57:13 +10:00
Damien Elmes
dc62d5072c use bytes for deck config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
164bd29438 use bytes for config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
5a0203c7b4 display warning when config load fails 2020-04-06 15:39:47 +10:00
Damien Elmes
34cca119e3 use blobs for config 2020-04-06 15:39:47 +10:00
Damien Elmes
676f4e74a8 store config in separate DB table 
- mtime is tracked on each key individually, which will allow
merging of config changes when syncing in the future
- added col.(get|set|remove)_config()
- in order to support existing code that was mutating returned
values (eg col.conf["something"]["another"] = 5), the returned list/dict
will be automatically wrapped so that when the value is dropped, it
will save the mutated item back to the DB if it's changed. Code that
is fetching lists/dicts from the config like so:

col.conf["foo"]["bar"] = baz
col.setMod()

will continue to work in most case, but should be gradually updated to:

conf = col.get_config("foo")
conf["bar"] = baz
col.set_config("foo", conf)
 2020-04-06 15:39:47 +10:00
Damien Elmes
0750ad5c62 fix duplicates search 2020-04-04 17:38:35 +10:00
Damien Elmes
7375a0389a add flag to skip downgrade on collection close 
Disabled for now; when enabled it will allow faster collection
open and close in the normal case, while continuing to downgrade
when exporting or doing a full sync.

Also, when downgrading is disabled, the journal mode is no longer
changed back to delete.
 2020-04-04 17:21:45 +10:00
Damien Elmes
45ea0ddefd use separate schema version for tag changes 
Will prevent issues if user upgrades after an unclean shutdown
 2020-04-04 10:14:51 +10:00
Damien Elmes
3b619c4ca5 use case folding when sorting in canonify 2020-04-03 19:34:46 +10:00
Damien Elmes
ac4284b2de update tag handling 
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
 2020-04-03 19:34:46 +10:00
Damien Elmes
333d0735ff preserve mtime/usn when syncing deck config, and add snake_case names 2020-04-03 19:34:46 +10:00
Damien Elmes
ee71e5dd59 use support URL and point to source repo 2020-04-03 08:44:59 +10:00
evandrocoan
069cc11a86 Created the GitHub Actions step Upload to PyPi to run only on 
tagged releases of anki:
https://anki.tenderapp.com/discussions/ankidesktop/40025-anki-in-python-package-index
 2020-04-02 02:57:34 -03:00
Damien Elmes
70a1992f70 fix is:due search 2020-04-02 09:33:05 +10:00
Damien Elmes
c68a2fc838 return interrupted err when media sync aborted 2020-04-01 21:40:14 +10:00
Damien Elmes
05cc47a5d3 fix push-i18n-ftl 2020-04-01 18:20:31 +10:00
Damien Elmes
546f4b44fc FString -> TR 2020-04-01 18:16:43 +10:00
Damien Elmes
dfa1ce6429 translate default deck config name 2020-04-01 17:49:25 +10:00
Damien Elmes
a5940e3557 add fixme 2020-04-01 17:36:33 +10:00
Damien Elmes
d396460332 ignore .DS_Store in trash 
https://anki.tenderapp.com/discussions/ankidesktop/40035-anki-2123-on-macos-ds_store-files-in-new-anki-trash
 2020-04-01 13:37:57 +10:00
Damien Elmes
6495240914 add some timeouts 
I'm waiting to hear back from the reqwests author on the best way to
handle this. For now, this change just adds short timeouts
to the quick requests, and will wait for up to an hour when sending/
receiving files to allow for the slow connection+large file case.
 2020-04-01 10:03:54 +10:00
Damien Elmes
c9da4bc1a6 allow aborting a media sync while http req in flight 
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
 2020-04-01 09:49:25 +10:00
Damien Elmes
35c03af520 split deck config into separate SQL table 
- on collection load, the schema is upgraded to 12
- on collection close, the changes are reversed so older clients
can continue to open the collection
- in the future, we could potentially skip the reversal except
when exporting/doing a full sync
- the same approach should work for decks, note types and tags in the
future too
- the deck list code needs updating to cache the deck confs for the
life of the call
 2020-03-30 20:01:16 +10:00
Damien Elmes
004cc2b5f8 move deck conf handling to backend 2020-03-30 14:39:46 +10:00
Damien Elmes
bf83715ee0 initial work on undo support 2020-03-29 17:52:16 +10:00
Damien Elmes
f90e5dbe2c remove usn cache 
SqliteStorage no longer contains any mutable state
 2020-03-29 12:50:38 +10:00
Damien Elmes
d1ecf33c72 cache timing_today in collection, update it when cutover reached 2020-03-29 12:26:24 +10:00
Damien Elmes
fdeca610b0 drop separate RequestContext/StorageContext 2020-03-29 12:12:35 +10:00
Damien Elmes
2810d3883b roll back the vec cache changes 
prepare_cached() is sufficiently fast, and allowing the vec cache
to persist across calls is complicated due to lifetime restrictions
 2020-03-29 09:26:09 +10:00
Damien Elmes
1a1a00d50f Revert "add benchmark for vec cache" 
This reverts commit 82ed288dc5.
 2020-03-29 09:16:11 +10:00
Damien Elmes
f6219b59ac
Merge pull request #528 from evandroforks/create_actions_for_windows_macos 
Create actions for Windows and Mac OS
 2020-03-27 15:26:22 +10:00
Damien Elmes
84b84ae31c use add_card() 2020-03-27 15:11:07 +10:00
Damien Elmes
6b9378fb41 add (unused) add_card, and move sql to separate files 2020-03-27 14:48:25 +10:00
Damien Elmes
82ed288dc5 add benchmark for vec cache 
test storage::sqlite::bench::bench_hash_cache ... bench:         399 ns/iter (+/- 27)
test storage::sqlite::bench::bench_no_cache   ... bench:       4,854 ns/iter (+/- 499)
test storage::sqlite::bench::bench_vec_cache  ... bench:           0 ns/iter (+/- 0)
 2020-03-27 09:59:48 +10:00
Damien Elmes
2f4e35d566 store cached statements in a vec instead of separate optionals 2020-03-27 09:51:19 +10:00
evandrocoan
b07454ca0c Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 2020-03-26 19:31:01 -03:00
Damien Elmes
369d2d89d9 update card in transaction 2020-03-27 07:58:25 +10:00
Damien Elmes
7f16f436c3 handle mtime/usn bump in backend, and tweak integer sizes 2020-03-26 20:55:04 +10:00
Damien Elmes
7a56d18539 report invalid type/queue 2020-03-26 19:32:35 +10:00
Damien Elmes
47504245dc update_card() 2020-03-26 18:54:20 +10:00
Damien Elmes
7ddaf93f7a use backend to get card 2020-03-26 17:47:43 +10:00
Damien Elmes
8abba00496 DeckID, CardID 2020-03-26 15:10:40 +10:00
Damien Elmes
f52e775354 NoteTypeID 2020-03-26 15:00:24 +10:00
Damien Elmes
4ec30e412a newtype NoteID 2020-03-26 14:42:43 +10:00
Damien Elmes
5eed3d7f71 use a macro for newtype defs 2020-03-26 13:50:20 +10:00
Damien Elmes
39f916a23e usn newtype 2020-03-26 13:06:02 +10:00
Damien Elmes
eb89a2db3f use newtypes for distinguishing between second and millisecond stamps 2020-03-26 12:59:51 +10:00
evandrocoan
721453f923 Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	.github/workflows/checks.yml
#	Makefile
#	pylib/Makefile
#	qt/Makefile
#	qt/tools/typecheck-setup.sh
#	rspy/Makefile
 2020-03-24 22:49:49 -03:00
Damien Elmes
01e9f889b7 add newline to mear clearly separate different ftl files 2020-03-25 08:53:42 +10:00
Damien Elmes
289bdde20c handle fields with embedded nuls 2020-03-24 14:53:33 +10:00
evandrocoan
20c9837770 Created the .github/workflows/windows.yml 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	Makefile
#	qt/tools/typecheck-setup.sh
#	react/Makefile
#	rspy/Makefile
#	svelte/Makefile
#	tslib/Makefile
 2020-03-23 19:51:03 -03:00
Damien Elmes
11a4d582b4 convert asc to desc instead of appending desc to the end of the order 
as the latter doesn't work when sorting on more than one column

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48174812
 2020-03-23 19:53:57 +10:00
Damien Elmes
14a970e923 rename long filenames in fields if files renamed in a previous sync 2020-03-23 18:38:45 +10:00
Damien Elmes
4e2e0d1b84 fix setting of wal 2020-03-23 13:52:57 +10:00
Damien Elmes
430f1ad616 handle trailing whitespace inside group 2020-03-22 18:52:44 +10:00
Damien Elmes
25ff4642ec accept now_mins_west for v2 legacy timing as well 2020-03-22 17:33:14 +10:00
Damien Elmes
69d8cdd9ed use backend for v1 and v2 cutoff calculations 
this should also fix the CI failures, which were happening because
the datetime module wasn't matching the shifted time.time()
 2020-03-22 14:28:26 +10:00
Damien Elmes
c5629e96df exclude autogenerated src from build deps 
prevents unnecessary rebuilds
 2020-03-22 03:39:38 +00:00
Damien Elmes
47fcdd0723 possible fix for CI failure 2020-03-22 13:17:00 +10:00
Damien Elmes
f28e57a367 add enum for controlling sort order 
eg col.find_cards("", order=BuiltinSortKind.CARD_DUE)
 2020-03-22 12:59:24 +10:00
Damien Elmes
199713a39a handle collections with sortBackwards set to 0 instead of a bool 2020-03-22 10:26:09 +10:00
Damien Elmes
cc44523449 remove debugging line 2020-03-21 18:29:04 +10:00
Damien Elmes
9696e959be increase the rated search cap to 365, and allow searches for ease 0 
An add-on appears to use ease 0 when rescheduling cards, and it may
make sense for Anki to do the same in the future as well.
 2020-03-21 16:05:17 +10:00