Commit graph

327 commits

Author SHA1 Message Date
Brett Smith
855c1c2bf0 books: Start Loader class. 2020-04-21 10:47:13 -04:00
Brett Smith
adf402442b books: Introduce Year type alias.
This is going to be repeated a lot as we write book-loading methods.
2020-04-21 09:51:27 -04:00
Brett Smith
51137815d3 config: fiscal_year_begin returns a FiscalYear. 2020-04-21 09:07:14 -04:00
Brett Smith
5c60666619 books: Start FiscalYear class. 2020-04-20 17:20:26 -04:00
Brett Smith
894f044093 config: Add Config.fiscal_year_begin() method. 2020-04-20 14:31:22 -04:00
Brett Smith
7f45788235 config: Start configuration file with books path.
Ultimately I would like to make it possible to configure the software
entirely through this file, rather than the hodgepodge system we have
now. But that can come later.
2020-04-12 22:27:52 -04:00
Brett Smith
b28646aa12 core.RelatedPostings: Add iter_with_balance method.
payment-report and accrual-report query to find the last date a
series of postings had a non/zero balance. This method is a good
building block for that.
2020-04-12 15:18:19 -04:00
Brett Smith
5aa30e5456 reports.core: Start Balance class. 2020-04-12 14:56:00 -04:00
Brett Smith
219cd4bc37 reports.core: Start RelatedPostings class. 2020-04-12 09:47:41 -04:00
Brett Smith
171aed16f9 data: Bugfix super() arguments in Amount.__new__.
Apparently I was more tired than I realized last night.
2020-04-12 08:58:27 -04:00
Brett Smith
01c3b975d8 data: Fix Amount.__new__.
See the comments for background and rationale.
2020-04-11 22:26:01 -04:00
Brett Smith
14a87e792b data: Move iter_posting into Posting class methods.
As I move into reporting code, having Posting.from_beancount() is
handy, and then from_txn() might as well come along for the ride.
2020-04-11 16:16:35 -04:00
Brett Smith
eb7f73e644 data.PostingMeta: Add date property.
This is something reporting tools will want a lot. This will make it
easier for them to look at just postings without worrying about the
parent transaction.
2020-04-11 09:20:35 -04:00
Brett Smith
a156617b4d config: Add Config.config_file_path() method. 2020-04-10 10:53:39 -04:00
Brett Smith
24813a9b81 config: Ignore non-absolute XDG_CACHE_HOME.
Per the spec.
2020-04-10 08:34:55 -04:00
Brett Smith
c7fbf5b5d5 config: s/XDG_CACHE_DIR/XDG_CACHE_HOME/g
Per the spec.
2020-04-10 08:24:34 -04:00
Brett Smith
9f0c30738d plugin: Most validations skip opening balance transactions. RT#10642. 2020-04-09 15:12:04 -04:00
Brett Smith
4eaba1ebf6 data: Add is_opening_balance_txn function. 2020-04-09 15:11:16 -04:00
Brett Smith
d66ba8773f data: Make balance_of currency-aware. 2020-04-09 14:13:07 -04:00
Brett Smith
c6dc2d83ac data.Amount: Introduce class and simplify code to use it.
See docstring for full rationale. This greatly reduces the need for other
plugin code to handle the case of `post.units.number is None`, eliminating
the need for entire methods and letting it do plain numeric comparisons.
2020-04-09 12:00:38 -04:00
Brett Smith
e00ec95d93 Account: Add is_opening_balance method. 2020-04-08 15:04:25 -04:00
Brett Smith
8f81530f6d meta_approval: Use data.balance_of. 2020-04-08 14:20:00 -04:00
Brett Smith
bb84cb5741 data.balance_of: Take account predicates, not just names.
For increased flexibility.
In particular, now you can pass in Account boolean methods to
call those directly.
2020-04-08 14:16:57 -04:00
Brett Smith
28e59e7a3b data: Add balance_of() function. 2020-04-08 11:55:00 -04:00
Brett Smith
212036b25e meta_approval: Stop checking payables.
That's handled by meta_payable_documentation now.
2020-04-07 15:45:29 -04:00
Brett Smith
ce34554bd4 meta_payable_documentation: Don't check Liabilites:Payable:Vacation. 2020-04-07 15:44:40 -04:00
Brett Smith
3dfe266945 meta_payable_documentation: Bugfix which metadata we're checking.
Checking approval/contract was in the original specification,
looking at invoice instead of approval was a pure brain fart.
2020-04-07 15:31:58 -04:00
Brett Smith
dd19e2a7a6 meta_payable_documentation: Start validation. RT#10643. 2020-04-07 15:29:15 -04:00
Brett Smith
f7bb036366 meta_receipt: Stop checking Assets:PayPal. RT#10637.
We only want to enforce paypal-id on these postings, and that's done with
the introduction of MetaPayPalID.
2020-04-07 14:47:02 -04:00
Brett Smith
21bea11beb meta_paypal_id: Start validator. RT#10260. 2020-04-07 14:41:06 -04:00
Brett Smith
0bf44ade7a plugin: Add HookRegistry.load_included_hooks() method.
This lets us import the plugin module without importing all of the included
hooks. This provides better isolation and error reporting in case there's
something like a syntax problem in one of the hooks: it doesn't cause
importing any plugin module to fail.
2020-04-07 13:31:09 -04:00
Brett Smith
fdb62dd1c6 plugin.core: _RequireLinksPostingMetadataHook can check several metadata.
Extend the base class from checking 1 metadata value to checking N.

This is preparation for RT#10643, letting payables be documented with
invoice or contract.

This does unify error reporting, because now we always report all
type/invalid value errors *plus* a missing error if appropriate.
I think this consistency and thoroughness is appropriate, although
it did require some adjustments to the tests.
2020-04-06 22:02:14 -04:00
Brett Smith
0413fed8b9 meta_entity: Use payee as entity when metadata not available. RT#10529. 2020-04-06 16:03:56 -04:00
Brett Smith
cbd50af302 rtutil: Explicit fspath cast to avoid a Py3.6 typecheck issue.
On Py3.6, either sqlite3.connect() doesn't take a path-like object,
or the type stubs don't know that.
2020-04-06 15:21:56 -04:00
Brett Smith
e1c507c025 plugin.core: Less introspective DIRECTIVE setting.
Python 3.6 does not implement __class_getitem__, and because of that
it's not possible to introspect when things like Hook[Transaction]
are called. Sidestep the issue with a more explicit assignment.
2020-04-06 15:18:54 -04:00
Brett Smith
6f3e5eb905 plugin.core: Clean goofy indirect import. 2020-04-06 14:58:21 -04:00
Brett Smith
600c9d9d6f meta_receipt: Correctly fall back to other metadata on zero-value postings.
RT#10633.
2020-04-05 15:29:04 -04:00
Brett Smith
4437a130d6 meta_receivable_docs: Not required for paid receivables. RT#10634. 2020-04-05 15:06:10 -04:00
Brett Smith
f24f941cfb meta_approval: Not required for bank transfers. RT#10635. 2020-04-05 14:49:39 -04:00
Brett Smith
bce438167c meta_approval: Required when payables are accrued. RT#10259. 2020-04-05 14:49:16 -04:00
Brett Smith
6658696d06 meta_receipt: Use check-id as fallback metadata for outgoing checks.
When we send checks, we don't have a check document anywhere (for
security reasons), we just note the check number. Update the
validation to match. RT#10507.
2020-04-04 10:54:08 -04:00
Brett Smith
c712105bed Revise chart of accounts used throughout.
The main impetus of this change is to rename accounts that were outside
Beancount's accepted five root accounts, to move them into that
structure. This includes:

  Accrued:*Payable: → Liabilities:Payable:*
  Accrued:*Receivable: → Assets:Receivable:*
  UneanedIncome:* → Liabilities:UnearnedIncome:*

Note the last change did inspire in a change to our validation rules. We no
longer require income-type on unearned income, because it's no longer
considered income at all. Once it's earned and converted to an Income
account, that has an income-type of course.

This did inspire another rename that was not required, but
provided more consistency with the other account names above:

  Assets:Prepaid* → Assets:Prepaid:*

Where applicable, I have generally extended tests to make sure one of each
of the five account types is tested. (This mostly meant adding an Equity
account to the tests.) I also added tests for key parts of the hierarchy,
like Assets:Receivable and Liabilities:Payable, where applicable.

As part of this change, Account.is_real_asset() got renamed to
Account.is_cash_equivalent(), to better self-document its purpose.
2020-04-03 10:34:10 -04:00
Brett Smith
499f18ff62 meta_entity: Adjust what entities are allowed based on today's books.
See the comments throughout for more discussion about what cases are
or aren't allowed, and why.
2020-04-01 13:38:37 -04:00
Brett Smith
90a58ef112 meta_project: Not required on Equity accounts. 2020-04-01 11:10:14 -04:00
Brett Smith
3a4c8526b2 meta_entity: Not required for Equity accounts. 2020-03-31 15:04:15 -04:00
Brett Smith
3aee1efdc1 data: Define __slots__ for core classes.
Because these are the classes that get instantiated many times while
iterating transactions, the performance benefit of defining __slots__
is worth the development overhead.
2020-03-31 15:00:15 -04:00
Brett Smith
fd2830f483 plugin: Load all existing hooks to registry. 2020-03-31 14:52:38 -04:00
Brett Smith
5566672cd6 plugin: Ensure run() can deal with all possible directives. 2020-03-31 14:30:49 -04:00
Brett Smith
87f35fe27f rtutil: Remove debug print. 2020-03-31 14:01:01 -04:00
Brett Smith
7a4e1c52f3 errors: InvalidMetadataError stores its key and value.
This makes it easier to introspect than trying to parse the message,
which is meant primarily for human readers.
2020-03-31 11:50:27 -04:00
Brett Smith
a9eab2d4ea meta_receivable_documentation: Start hook. 2020-03-31 11:42:21 -04:00
Brett Smith
1fc9363b26 data: Add is_credit() and is_debit() methods to Posting.
The main motivation for this change is to make sure that higher-level
code deals with the fact that self.units.number can be None, and has
an easy way to do so.

I'm not sure all our code is *currently* doing the right thing for this
case, because I'm not sure it will ever actually come up. It's possible
that earlier Beancount plugins fill in decimal amounts for postings
that are originally loaded with self.units.number=None. I'll have to see
later whether this case comes up in reality, and then deal with it if so.
For now the safest strategy seems to be that most code should operate
when self.units.number is None.
2020-03-31 10:07:25 -04:00
Brett Smith
a0a3b04e50 meta_receipt: Accept paypal-id in lieu of receipt for PayPal credits. 2020-03-30 15:48:19 -04:00
Brett Smith
381160f0de meta_receipt: Accept invoice in lieu of receipt for credit card charges. 2020-03-30 15:48:19 -04:00
Brett Smith
7cac21b780 meta_receipt: Applies to both credits and debits. 2020-03-30 15:47:30 -04:00
Brett Smith
5a5caf0ddc meta_receipt: Clean unused import. 2020-03-30 15:16:04 -04:00
Brett Smith
2d49f7dfbc data: Add Account.is_checking() method. 2020-03-30 15:01:25 -04:00
Brett Smith
959bda307b typing: Refine any generics when possible.
These were tightened up by running mypy with --disallow-any-generics
and cleaning up reported errors.

There are still a couple of cases I don't know how to deal with
right now that prevent me from setting that option permanently:

* plugin/__init__.py sets up a dict where the key is a Directive
  and its value is a set of hooks that correspond to that type
  of directive. The relationship between key and value is not
  expressable through the type system, and any other solution would
  make the code way more involved and less dynamic.

* The type variable used in _GenericRange still isn't quite right,
  in the sense that specifying a value for CT creates more errors
  I don't know how to deal with. Protocols seem like the right
  solution for that but they apparently didn't land for Python 3.7
  that I can see.
2020-03-29 23:28:03 -04:00
Brett Smith
22d5b7e90a setup: Disallow untyped calls. 2020-03-29 23:22:35 -04:00
Brett Smith
747ef25da6 setup: Disallow untyped defs.
Mostly this meant giving annotations to low-value functions like
the error classes and __init_subclass__, but it's worth it for
the future strictness+documentation value.
2020-03-29 23:18:40 -04:00
Brett Smith
e6894c2b46 setup: Enable stricter type checking.
This caught the "return instead of raise" bug in meta_project.
2020-03-29 19:39:09 -04:00
Brett Smith
d9dca2cd68 data: Posting.is_payment casts threshold to address typing issue. 2020-03-29 15:32:51 -04:00
Brett Smith
d8c7242208 meta_approval: Pass configured payment_threshold to is_payment. 2020-03-29 14:41:57 -04:00
Brett Smith
043644d194 meta_approval: Start hook. 2020-03-29 11:14:51 -04:00
Brett Smith
30d371278a plugin: Refactor hooks to use new payment-related methods. 2020-03-29 10:30:54 -04:00
Brett Smith
5f85d9c747 config: Add Config.payment_threshold() method.
This just returns a constant for now, but we know it may need to be
configurable in the future.  Other code can start using this now
to be configurable in the future.
2020-03-29 10:21:37 -04:00
Brett Smith
93feb2f4a3 data: Add Posting.is_payment() method. 2020-03-29 10:18:51 -04:00
Brett Smith
2909c405e6 meta_receipt: Start hook. 2020-03-28 22:19:49 -04:00
Brett Smith
7658aaedf8 plugin: Refactor _RequireLinksPostingMetadataHook out of MetaInvoice.
This will be a common hook type.
2020-03-28 14:48:49 -04:00
Brett Smith
d9420ac2b6 meta_invoice: Start hook. 2020-03-28 14:31:17 -04:00
Brett Smith
46cfc558ec plugin: Link checkers use Metadata class. 2020-03-28 13:36:56 -04:00
Brett Smith
9b63d898af data: Add Metadata class.
As I start writing more link-checking hooks, I want a common place to
write link-parsing code.  This new class will be that place.
2020-03-28 13:35:38 -04:00
Brett Smith
2cb131423f errors: Redo InvalidMetadataError.
This needs to be generally usable for transactions.
2020-03-28 09:47:40 -04:00
Brett Smith
33cb734b19 config: Create cache database files with 0o600 mode.
I wasn't too worried about this earlier because the cache mainly stores
a bunch of numbers, but there's a little more than that: the generated
URLs also include original attachment filenames, which might be sensitive
(referencing people's names, bank names, etc.). Tighten security
accordingly.
2020-03-27 10:55:30 -04:00
Brett Smith
a8407c7b6a rtutil: Add RTLinkCache class to cache links to disk.
This will greatly reduce RT requests across multiple runs
and speed up link checking/conversion.
2020-03-27 07:35:45 -04:00
Brett Smith
1155212376 config: Add Config.cache_dir_path method. 2020-03-25 15:15:38 -04:00
Brett Smith
e8e7137216 rtutil: Fix parsing server URLs that include /REST/. 2020-03-25 10:18:09 -04:00
Brett Smith
a3630ab04e rtutil: Add docstring with basic rationale. 2020-03-25 10:18:01 -04:00
Brett Smith
2fb7826ab4 rtutil: Clean unused code. 2020-03-25 10:12:50 -04:00
Brett Smith
a61b74308f rtutil: Make RT.parse a classmethod.
Because it works, and I can imagine it being useful to have this
method more handy to other code.
2020-03-25 00:13:18 -04:00
Brett Smith
4874a107e8 meta_rt_links: Start hook. 2020-03-25 00:12:20 -04:00
Brett Smith
c4ce59da75 config: Add Config.rt_wrapper method.
This returns a cached version of the new rtutil.RT class,
so that a single process can easily grab references to one
instance with one cache of ticket/attachment data, etc.
2020-03-24 23:30:31 -04:00
Brett Smith
9fbc658aa6 rtutil: Add RT.parse method.
This method can parse the ticket and (optional) attachment IDs
out of our short-hand link formats, both for Ledger and Beancount.
2020-03-24 18:24:31 -04:00
Brett Smith
d5a6141f6d rtutil: Start module.
For now, this is basically the Python version of
ledger-tag-convert.plx.  It knows how to create RT web links from
ticket and attachment IDs.  It confirms that those objects actually
exist too.  It may grow to encompass other functionality in the
future.
2020-03-24 17:23:54 -04:00
Brett Smith
f09f029fc4 config: Add Config.rt_client method. 2020-03-24 09:08:08 -04:00
Brett Smith
8d3816a8fd config: Add Config.rt_credentials method.
This loads settings from the same environment variables and ~/.rtrc
file as the rt CLI.

Note that it does *not* support RTCONFIG and the config file
searching, because right now that seems like more work for more
trouble to me.
2020-03-23 15:19:15 -04:00
Brett Smith
d845332c7e meta_entity: Clean unused imports. 2020-03-21 13:55:42 -04:00
Brett Smith
ad268f049d meta_entity: Start hook. 2020-03-21 13:53:33 -04:00
Brett Smith
5c6043311b meta_repo_links: Start hook. 2020-03-20 16:47:06 -04:00
Brett Smith
1500d2d9d8 meta_project: Start hook. 2020-03-20 12:08:10 -04:00
Brett Smith
0d370c445b plugin: User configuration is passed to hooks on initialization. 2020-03-19 17:23:27 -04:00
Brett Smith
84d8adb7f6 plugin: Move hook initialization from HookRegistry to run().
Makes more sense here so run can report errors in hook configuration.
2020-03-19 16:42:13 -04:00
Brett Smith
71531913d5 config: Start module.
Ultimately all values that come from user configuration will be generated
here.
2020-03-19 14:31:21 -04:00
Brett Smith
a78db2ed36 data: Add docstrings. 2020-03-19 12:19:34 -04:00
Brett Smith
4fee91ad48 data.Account: Use multi-arg is_under in is_real_asset. 2020-03-19 10:43:20 -04:00
Brett Smith
7cd569be7b data.Account: Add is_income method. 2020-03-19 10:23:55 -04:00
Brett Smith
b7089f5ddf plugin: Add meta_income_type validator. 2020-03-19 09:52:43 -04:00
Brett Smith
9b8563f3f0 data.Account: is_under accepts multiple arguments. 2020-03-19 09:47:10 -04:00
Brett Smith
cbd94d9b75 core: Make MetadataEnum aliases optional. 2020-03-18 18:23:42 -04:00
Brett Smith
89aaae821b data.Account: Exclude PrepaidVacation from "real" assets.
Same rationale as PrepaidExpenses.
2020-03-18 16:25:37 -04:00
Brett Smith
3f9e67de3a meta_tax_implication: Don't enforce on Assets:PrepaidExpenses. 2020-03-18 09:00:47 -04:00
Brett Smith
28238643a3 data.Account: Add is_real_asset method. 2020-03-18 08:59:03 -04:00
Brett Smith
661fe676e1 Document known typecheck failures. 2020-03-18 08:24:07 -04:00
Brett Smith
b2ef561c85 data: Introduce Posting class.
Our version of Posting is interface-compatible with Beancount's,
but makes stronger guarantees about the data types for our
higher-level code to rely on.
2020-03-18 07:48:08 -04:00
Brett Smith
163ecbc7d3 data: iter_postings uses Account. 2020-03-17 18:06:43 -04:00
Brett Smith
6c0f23b2fa data: Start Account class. 2020-03-17 18:05:24 -04:00
Brett Smith
3a73bc2239 data: Start iter_postings function. 2020-03-17 17:08:47 -04:00
Brett Smith
f4978046b8 data: Start module with PostingMeta class.
As I'm pushing to reimplement payment-report/income-report, I see query and
reporting tools will want easy access to these kinds of views, so here it
starts.
2020-03-17 17:08:05 -04:00
Brett Smith
3fbc14d377 Improve organization between modules.
* Rename _typing to beancount_types to better reflect what it is.
* LessComparable isn't a Beancount type, so move that to
  plugin.core with its dependent helper classes.
* Errors are a core Beancount concept, so move that module to the
  top level and have it include appropriate type definitions.
2020-03-15 16:03:57 -04:00
Brett Smith
a41feb94b3 plugin: Transform posting hooks into transaction hooks.
I feel like posting hooks a case of premature optimization in early
development. This approach reduces the number of special cases in
the code and allows us to more strongly reason about hooks in the
type system.
2020-03-15 15:50:14 -04:00
Brett Smith
c9ff4ab746 plugin: Settle on words-with-dashes metadata keys. 2020-03-15 10:36:49 -04:00
Brett Smith
ee038d7b7d plugin.core: Add type hints. 2020-03-08 18:24:51 -04:00
Brett Smith
547ae65780 plugin.core: _meta_set properly handles when post.meta is None.
post is a NamedTuple, so attribute assignment is not allowed.
Instead we have to construct a whole new Posting.
2020-03-08 11:32:03 -04:00
Brett Smith
c3c19b0ad2 plugin.core: Document base classes.
Since so much other code is built on these, I wanted to make sure I
wrote this while it was all fresh in my head.
2020-03-06 10:37:54 -05:00
Brett Smith
6e0c31f0ab plugin.core: Add missing import. 2020-03-06 09:24:15 -05:00
Brett Smith
d34db71542 plugin: Introduce HookRegistry.
This is the layer that keeps track of the different groups of hooks and
can filter them before runtime. The idea here is that you'll be able
to do things like skip hooks that require network access when you don't
have it, or skip CPU-intensive hooks when you don't need them, etc.
2020-03-06 09:22:55 -05:00
Brett Smith
e9e2bb9b00 plugin: Rename the main method of hooks from check to run.
This will be more appropriate when we have hooks that do more than
check metadata.
2020-03-05 17:48:59 -05:00
Brett Smith
53329c7a23 plugin: Start entry point.
This doesn't integrate with existing hooks but at least sketches out the
main loop.
2020-03-05 17:41:58 -05:00
Brett Smith
dbe9362987 meta_tax_implication: Initial module and tests. 2020-03-05 15:48:10 -05:00
Brett Smith
5849b562d0 expenseAllocation: Switch to own MetadataEnum class.
Python's own enum works fine for the simple values that expenseAllocation
uses, but it won't work as well for metadata like taxImplication where
many of the values aren't legal Python identifiers. Introduce our own
MetadataEnum class with the necessary functionality, and switch to that.
2020-03-05 15:33:27 -05:00
Brett Smith
16c47c64b2 expenseAllocation: Date-limit the transactions we work on.
This prevents the plugin from giving meaning to postings that
might not really be there.
2020-03-05 14:37:47 -05:00
Brett Smith
910b95c942 errors: Add a __repr__.
This makes the errors easier to understand in test failures.
2020-03-05 13:57:20 -05:00
Brett Smith
34090df23f expenseAllocation: Set default values by account. 2020-03-05 13:44:36 -05:00
Brett Smith
99f97d3f58 expenseAllocation: Accept 'admin' shorthand for 'administration'. 2020-03-05 12:23:03 -05:00
Brett Smith
d436a388f7 expenseAllocation: Only check Expenses postings. 2020-03-05 12:10:05 -05:00
Brett Smith
7862919022 expenseAllocation: Start checker.
This is the simplest version of a common validation we're going to do:
make sure that a particular piece of metadata has one of a set of
values.

This checker needs some bounds checking but I wanted to err on the
side of committing this early because it introduces so much base
infrastructure.
2020-03-05 12:05:13 -05:00
Brett Smith
c5dd7984bc Initial commit: license, setup.py, gitignore, skeleton structure. 2020-03-05 09:33:23 -05:00