Everything it said was a problem has been done legitimately in our books at
one point or another.
* Variation in contract can happen in different line items of an invoice or
"group of contractor" situations.
* Variation in cost can happen because one invoice spans a period of time,
like donation matching programs. There is probably still value in a tool
that checks to make sure we use consistent rates each day, but that
affects all kinds of transactions, not just accruals, so it would be
done better in a separate tool.
* Variation in account happens because invoices legitimately span accrual
accounts, like donation matching programs with fees payable.
So: it's gone, good riddance.
This adds almost all the metadata that's relevant to accruals.
I considered adding statement, but that cuased rows to get spaced out a lot,
and statement's kind of a low-value column, so I decided against it.
Ultimately I would like to make this configurable but that's for the
future.
It is more common than I realized that we split an invoice by
entity on the accrual side, so this supports that better.
It still disregards inconsistency between accrual entity and payment entity
for reporting purposes, to help keep reporting clean around automatic
imports.
The changes to BaseReport._report shook out because at this point, the group
key is effectively arbitrary and shouldn't be used for any reporting
purposes.
This works fine with how we're currently using it, makes transformation
methods easier to implement, and avoids potential bugs where a balance is
initialized with a bad mapping.
So far we've been implicitly relying on this by the user passing search
terms that filter out the opening balance transaction. That will stop
happening with the aging report, so we need to do it ourselves.
This was an early mistake, it makes data consistency mistakes too
easy, and I only used it once so far in actual code. Going to fix
this now so I can more safely build on top of this data structure.
It turns out the provided implementation gets us most of the way there,
we just needed to add handling for the special case of zero balances.
Now it's confirmed with tests.
The test changes make them order-sensitive, which they should be.
It's important that our loader methods return date-sorted entries
just like Beancount itself would.
This makes the output more useful for broad searches like on an
entity. Invoices that cross FY boundaries will appear to be paid
without being accrued, and so would appear when we were just
filtering zeroed-out invoices.
If we integrate the aging report into this module in the future,
that'll need to follow different logic, and just filter out
zeroed-out invoices. But the basic balance report and outgoing
report are more workaday tools, where more filtering makes them
more useful.
It makes sense to let the bookkeeper skip validations in situations
where the metadata requires information that might not be available
when entered. It does not make sense to skip validations that *must*
be available and affect the structure of the books, like project and
entity.
This commit ensures every plugin hook has a test for flagged
transactions, even for hooks that currently have the desired
behavior where no code changes were required for the test to
pass.
Building a string and loading it means Beancount can never cache any
load. It only caches top-level file loads because options in the
top-level file can change the semantics of included entries.
Instead use load_file as much as possible, and filter entries as
needed.
The old loading strategy didn't load options, which yielded some
spurious errors. It also created awkward duplication of plugin
information in the code as well as the books.
Implement a new loading strategy that works by reading one of the
"main files" under the books/ subdirectory and includes entries
for additional FYs beyond that.
This is still not ideal in a lot of ways. In particular, Beancount can't
cache any results, causing any load to be slower than it theoretically could
be. I expect more commits to follow. But some of them might require
restructuring the books, and that should happen separately.
This module basically reimplements the old payment-report+income-report,
in a single tool (after setting aside some of the checks that have moved
to the plugin). The aging report can be implemented here too when we need
that.
This makes methods like _replace available in real code, and caught the
bug where we can't use @functools.lru_cache with Transaction arguments,
because they're unhashable due to their mutable members.
Ultimately I would like to make it possible to configure the software
entirely through this file, rather than the hodgepodge system we have
now. But that can come later.
payment-report and accrual-report query to find the last date a
series of postings had a non/zero balance. This method is a good
building block for that.
This is something reporting tools will want a lot. This will make it
easier for them to look at just postings without worrying about the
parent transaction.
See docstring for full rationale. This greatly reduces the need for other
plugin code to handle the case of `post.units.number is None`, eliminating
the need for entire methods and letting it do plain numeric comparisons.
This lets us import the plugin module without importing all of the included
hooks. This provides better isolation and error reporting in case there's
something like a syntax problem in one of the hooks: it doesn't cause
importing any plugin module to fail.
Extend the base class from checking 1 metadata value to checking N.
This is preparation for RT#10643, letting payables be documented with
invoice or contract.
This does unify error reporting, because now we always report all
type/invalid value errors *plus* a missing error if appropriate.
I think this consistency and thoroughness is appropriate, although
it did require some adjustments to the tests.
When we send checks, we don't have a check document anywhere (for
security reasons), we just note the check number. Update the
validation to match. RT#10507.
The main impetus of this change is to rename accounts that were outside
Beancount's accepted five root accounts, to move them into that
structure. This includes:
Accrued:*Payable: → Liabilities:Payable:*
Accrued:*Receivable: → Assets:Receivable:*
UneanedIncome:* → Liabilities:UnearnedIncome:*
Note the last change did inspire in a change to our validation rules. We no
longer require income-type on unearned income, because it's no longer
considered income at all. Once it's earned and converted to an Income
account, that has an income-type of course.
This did inspire another rename that was not required, but
provided more consistency with the other account names above:
Assets:Prepaid* → Assets:Prepaid:*
Where applicable, I have generally extended tests to make sure one of each
of the five account types is tested. (This mostly meant adding an Equity
account to the tests.) I also added tests for key parts of the hierarchy,
like Assets:Receivable and Liabilities:Payable, where applicable.
As part of this change, Account.is_real_asset() got renamed to
Account.is_cash_equivalent(), to better self-document its purpose.
The main motivation for this change is to make sure that higher-level
code deals with the fact that self.units.number can be None, and has
an easy way to do so.
I'm not sure all our code is *currently* doing the right thing for this
case, because I'm not sure it will ever actually come up. It's possible
that earlier Beancount plugins fill in decimal amounts for postings
that are originally loaded with self.units.number=None. I'll have to see
later whether this case comes up in reality, and then deal with it if so.
For now the safest strategy seems to be that most code should operate
when self.units.number is None.
I'm getting to ready to extend this hook to deal with income receipts as
well as expense receipts. These changes let me write the tests in a more
declarative style, so I don't have to duplicate them all again to test
credits as well as debits.
Note that we're only testing debits right now, just like the existing tests,
because the default check_type for check() is PostType.DEBIT. Part of making
the changes will be changing that to PostType.BOTH.
This just returns a constant for now, but we know it may need to be
configurable in the future. Other code can start using this now
to be configurable in the future.
I wasn't too worried about this earlier because the cache mainly stores
a bunch of numbers, but there's a little more than that: the generated
URLs also include original attachment filenames, which might be sensitive
(referencing people's names, bank names, etc.). Tighten security
accordingly.
This returns a cached version of the new rtutil.RT class,
so that a single process can easily grab references to one
instance with one cache of ticket/attachment data, etc.
For now, this is basically the Python version of
ledger-tag-convert.plx. It knows how to create RT web links from
ticket and attachment IDs. It confirms that those objects actually
exist too. It may grow to encompass other functionality in the
future.
This loads settings from the same environment variables and ~/.rtrc
file as the rt CLI.
Note that it does *not* support RTCONFIG and the config file
searching, because right now that seems like more work for more
trouble to me.
As we start reading more sensitive user configuration (like RT credentials),
this fixture makes it simpler to ensure that the tests never read the user's
real configuration, which could lead to problems like passwords in tracebacks.
