NPO-Accounting/import2ledger
6
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
97 commits 1 branch 0 tags 208 KiB
Commit graph

62 commits

Author SHA1 Message Date
Brett Smith
2de14e4573 stripe: Add payouts importer. 2018-01-25 17:59:19 -05:00
Brett Smith
f3e8f774c7 stripe: Extract more columns from payment data. 2018-01-25 12:11:58 -05:00
Brett Smith
5681545267 nbpy2017: Bugfix entry data key name for custom Ledger template. 2018-01-24 18:01:16 -05:00
Brett Smith
37563ffae0 brightfunds: New importer. 
This commit adds infrastructure to treat XLS files like CSV files, and
import them using the base classes that already exist for that.
 2018-01-21 17:04:37 -05:00
Brett Smith
db59d2fc8c hooks.ledger_entry: Look up templates dynamically. 
If there's a 'ledger entry' key in the entry data, use that value as the
name of the template to load.  Thanks to this, nbpy2017 could collapse
multiple importers into one.

Otherwise, build a default template name based on the importer source, and
try to use that.

All the configuration names now end with "ledger entry" instead of starting
with "template".  This makes it clearer what they're for, in case we
support other kinds of output templates in the future.

I ended up changing the names of some of the importers so the default
template name was nice, rather than specifying template names for all of
them, to reduce the amount of name discrepancies across the codebase.
 2017-12-31 18:52:30 -05:00
Brett Smith
475de6db4e Clean stray file from initial development. 2017-12-31 17:58:47 -05:00
Brett Smith
cdec3d9aab hooks.ledger_entry: Bring in all Ledger-specific code. 
This includes the old "template" module, plus the associated
template-loading code from config.
 2017-12-31 17:29:14 -05:00
Brett Smith
6d1a7cb57d config: Make get_section a public method. 
As hooks and importers both require more configuration, they're gonna need
to have lower-level access to configuration settings.
 2017-12-31 16:50:07 -05:00
Brett Smith
76f2707aac hooks.ledger_entry: New hook to output Ledger entries. 
This is roughly the smallest diff necessary to move output to a hook.
There's a lot of code reorganization that should still happen to bring it
better in line with this new structure.
 2017-12-31 12:35:20 -05:00
Brett Smith
d2f8772e08 config: Add open_output_file method. 
Make this functionality accessible to hooks.
 2017-12-31 11:10:49 -05:00
Brett Smith
efe5768941 main: Set up a decimal context that safely handles currency. 2017-12-31 10:06:57 -05:00
Brett Smith
85b665200c hooks: Hooks declare what order they run in. 2017-12-30 18:11:41 -05:00
Brett Smith
e8bcbd5f99 hooks.add_entity: Remove common company suffixes from entity tags. 2017-12-30 17:29:54 -05:00
Brett Smith
2c6211c9ea main: Ensure the source_dir variable is a string. 2017-12-30 10:18:33 -05:00
Brett Smith
cdedd0fc7c importers: Add CSVImporterBase documentation for 0f4f83e. 2017-12-30 10:14:08 -05:00
Brett Smith
25c6454e2d importers.amazon: Initial version. 2017-12-27 13:44:35 -05:00
Brett Smith
11eccb60dd strparse: Fix default limit argument in split functions. 
The underlying string methods don't accept None.
 2017-12-27 13:35:52 -05:00
Brett Smith
0f4f83e079 importers._csv: Refactor header reading into CSVImporterBase. 2017-12-27 13:14:12 -05:00
Brett Smith
1b1e2d038c importers.benevity: Initial version. 2017-12-27 11:54:34 -05:00
Brett Smith
f2c869d783 hooks.add_entity: Add a corp_entity from a corporation variable. 2017-12-27 11:51:14 -05:00
Brett Smith
82e4bf5bd8 hooks.add_entity: Add name prefixes: der, la 2017-12-27 11:50:58 -05:00
Brett Smith
56cba1456f hooks.add_entity: Maintain sequences of name prefixes. 2017-12-27 11:50:37 -05:00
Brett Smith
07d22418d4 main: Provide additional template variables about each source path. 2017-12-27 11:39:56 -05:00
Brett Smith
40552d665a template: Let users split an account multiple times per template. 2017-12-27 11:39:34 -05:00
Brett Smith
9f720527f1 template: Bugfix newline handling. 
If a rendering ended with a zeroed out split, the output would contain an
extraneous trailing newline.  Rearrange the code to print a newline
preceding each content line.
 2017-12-27 11:36:25 -05:00
Brett Smith
e758db31ad hooks.add_entity: Don't try to rearrange non-Latin names. 2017-12-27 10:45:18 -05:00
Brett Smith
87f3209101 strparse: Improve extra text parsing in currency_decimal. 
This allows a symbol and a currency code to be in different parts of the
string, as long as there's at most one of each.
 2017-12-19 12:43:59 -05:00
Brett Smith
0f945c28df main: Clean noop line. 
I guess this was a remnant when I was trying to implement date filtering
before I decided to do it with hooks.
 2017-12-19 10:20:35 -05:00
Brett Smith
18eebbc0ed hooks: run() return value controls processing of entry data. 
Instead of using in-band signaling with the entry_data dict.
I don't know why I didn't think of this in the first place.
 2017-12-19 10:19:56 -05:00
Brett Smith
cc8c03ab62 main: Provide template variables about the file being imported. 2017-12-19 09:13:44 -05:00
Brett Smith
122323853d importers: Prefer using ChainMaps to copying and updating dicts. 2017-12-19 09:13:44 -05:00
Brett Smith
b41a7a99af hooks: Filter entries by setting entry_data['_hook_cancel'] to True. 
The current method only works for plain dicts and other simple mappings.
Mapping that may still contain items after .clear(), like ChainMap, can't
rely on the old method.
 2017-12-19 09:13:44 -05:00
Brett Smith
34b71baaf7 template: render() takes a single mapping argument. 
This lets it receive other mapping types like ChainMap.
 2017-12-19 09:13:00 -05:00
Brett Smith
1b98ab2015 util: Rename to strparse. 
Better describes what the functions in the module do.
 2017-12-19 06:42:05 -05:00
Brett Smith
6ea28c2c89 util: Add parse_currency_dec. 
The current importers trim lots of extraneous symbols and whitespace from
currency strings before passing them to Decimal().  This function takes care
of all that in a single place.
 2017-12-18 23:04:05 -05:00
Brett Smith
0734b6f7a5 hooks.invoice_payment: New hook. 
In the long run, this should be some kind of configurable hook to run
arbitrary regexps on arbitrary entry fields, and then on a match, add
another field with a format string using the match data.  But this is
reasonably generic for now.
 2017-12-18 17:40:30 -05:00
Brett Smith
9472be10f1 main: Open files with errors='replace'. 
Stripe payment CSVs have apparently random encoding, and no way what to tell
what they're using beyond chardet.

The better way to fix this is following the existing to-do item, opening
files in binary mode and creating an importer base class to provide text
wrapping.  This would let each importer be as strict as possible.
 2017-12-18 17:38:55 -05:00
Brett Smith
68a0a9625c importers.stripe: Generate payment entries for refunded payments. 
In order to be refunded, the payment had to be paid in the first place, so
we should still generate an entry for it.
 2017-12-18 17:32:33 -05:00
Brett Smith
4796932cb7 importers.patreon: Handle commas in large amounts. 2017-12-18 14:09:54 -05:00
Brett Smith
fea64ee138 main: Don't use importers whose template isn't configured. 
This lets users import only some information from sources that provide
multiple transaction streams at once.
 2017-12-18 12:35:50 -05:00
Brett Smith
0b665d388e template: Add is_empty method. 
This makes it easy to detect when a user has specifically cleared a
template; i.e., from a specific configuration section.
 2017-12-18 12:35:18 -05:00
Brett Smith
474a0390e3 template: Support custom payee lines. 2017-12-18 12:23:18 -05:00
Brett Smith
668906b944 importers.nbpy2017: Yield entries for the original invoice. 
This lets us write up a cleaner separation between the original act of
invoicing and the Stripe payment.
 2017-12-18 10:47:04 -05:00
Brett Smith
c98f0d9ff3 importers.nbpy2017: Refactor out _strpdate method. 2017-12-18 10:14:13 -05:00
Brett Smith
8da3fd4bd3 util: Add common word-slicing functions. 2017-12-18 10:14:01 -05:00
Brett Smith
e249049fc1 util: Move module loader functions to a new dynload module. 2017-12-18 09:26:22 -05:00
Brett Smith
f56571219b importers.nbpy2017: Don't deal with refunds. 
Only one refund was actually issued, so we can deal with that manually.  But
the code did its job of revealing those cases and checking we were handling
them well.
 2017-12-18 09:21:31 -05:00
Brett Smith
9d9254b69a template: Nicer formatting of account split lines. 
This provides nicer output when the whole line can fit in the desired width,
even if the account name or amount are unusually long.
 2017-12-17 21:05:22 -05:00
Brett Smith
93fffe6666 importers: New importer for North Bay Python invoices. 
This probably has no use outside Conservancy.
 2017-12-17 20:46:00 -05:00
Brett Smith
e3ec03cf19 template: Don't output accounts that end up having no amount. 2017-12-16 13:10:49 -05:00