Basically this behavior is an extension of the fact that the outgoing report
is grouped by RT ticket rather than "accrual data." Ripping this
functionality out of other reports was correct, but it needed to stay for
the outgoing report.
Now that we're accepting classifications, it's possible to specify account
options that select some but not all accounts at the same level of the
hierarchy. This commit tracks requested account names separately from sheet
names to do that correctly.
This required keeping the balances from write_row, and then a lot of other
changes followed from that. In particular it makes more sense to build the
fund report sheet from scratch rather than copying the breakdowns report and
chiseling the fund report out of it.
Some readers care about recent accruals, some don't. This presentation
accommmodates both audiences, providing the data while making it easy to
ignore or filter out recent accruals.
Now that make_consistent is really robust, there's much less need to do all
the consistency checking that was done in AccrualPostings.__init__. I expect
this will provide a performance benefit for large reports, since we'll be
calculating data for many fewer accrual groups. The only performance penalty
I see is that the aging report has to calculate the balance three times for
each row it reports, twice in write() and once in write_row(), but that
seems okay and can be cached separately if needed.
This accommodates cases of contracts without separate invoices,
where a series of payments are scheduled over time.
The dance we used to do of group-by-invoice, then make consistent was
already suspect. It was originally motivated by the consistency checks that
are now gone. Use this opportunity to clean up and just make make_consistent
a classmethod.
I wrote the changes to Balance.format() before the dependent changes to
Balance.copy(), so I was sort of counting on them to be implicitly
tested. But they should be explicit.
The period totals were reporting the balance of all the loaded postings, not
just the ones in the reporting date range.
Like the accrual report, introduce a RelatedPostings subclass that records
and saves all the information we need at group definition time, to help us
get it consistently right rather than redoing the same math over and over.
The ledger report wants to use this functionality, so make it available in a
higher-level module.
I took the opportunity to clean up a lot of the surrounding type
declarations. It is less flexible, since it relies on the static list of
types in RangeT, but I don't think the other method actually worked at all
except by cheating with generic Any.
Everything it said was a problem has been done legitimately in our books at
one point or another.
* Variation in contract can happen in different line items of an invoice or
"group of contractor" situations.
* Variation in cost can happen because one invoice spans a period of time,
like donation matching programs. There is probably still value in a tool
that checks to make sure we use consistent rates each day, but that
affects all kinds of transactions, not just accruals, so it would be
done better in a separate tool.
* Variation in account happens because invoices legitimately span accrual
accounts, like donation matching programs with fees payable.
So: it's gone, good riddance.
This adds almost all the metadata that's relevant to accruals.
I considered adding statement, but that cuased rows to get spaced out a lot,
and statement's kind of a low-value column, so I decided against it.
Ultimately I would like to make this configurable but that's for the
future.
It is more common than I realized that we split an invoice by
entity on the accrual side, so this supports that better.
It still disregards inconsistency between accrual entity and payment entity
for reporting purposes, to help keep reporting clean around automatic
imports.
The changes to BaseReport._report shook out because at this point, the group
key is effectively arbitrary and shouldn't be used for any reporting
purposes.
This works fine with how we're currently using it, makes transformation
methods easier to implement, and avoids potential bugs where a balance is
initialized with a bad mapping.
So far we've been implicitly relying on this by the user passing search
terms that filter out the opening balance transaction. That will stop
happening with the aging report, so we need to do it ourselves.
This was an early mistake, it makes data consistency mistakes too
easy, and I only used it once so far in actual code. Going to fix
this now so I can more safely build on top of this data structure.
It turns out the provided implementation gets us most of the way there,
we just needed to add handling for the special case of zero balances.
Now it's confirmed with tests.
The test changes make them order-sensitive, which they should be.
It's important that our loader methods return date-sorted entries
just like Beancount itself would.
This makes the output more useful for broad searches like on an
entity. Invoices that cross FY boundaries will appear to be paid
without being accrued, and so would appear when we were just
filtering zeroed-out invoices.
If we integrate the aging report into this module in the future,
that'll need to follow different logic, and just filter out
zeroed-out invoices. But the basic balance report and outgoing
report are more workaday tools, where more filtering makes them
more useful.
It makes sense to let the bookkeeper skip validations in situations
where the metadata requires information that might not be available
when entered. It does not make sense to skip validations that *must*
be available and affect the structure of the books, like project and
entity.
This commit ensures every plugin hook has a test for flagged
transactions, even for hooks that currently have the desired
behavior where no code changes were required for the test to
pass.
Building a string and loading it means Beancount can never cache any
load. It only caches top-level file loads because options in the
top-level file can change the semantics of included entries.
Instead use load_file as much as possible, and filter entries as
needed.
The old loading strategy didn't load options, which yielded some
spurious errors. It also created awkward duplication of plugin
information in the code as well as the books.
Implement a new loading strategy that works by reading one of the
"main files" under the books/ subdirectory and includes entries
for additional FYs beyond that.
This is still not ideal in a lot of ways. In particular, Beancount can't
cache any results, causing any load to be slower than it theoretically could
be. I expect more commits to follow. But some of them might require
restructuring the books, and that should happen separately.
