Any way to limit history processing for imports?
Posted: 19 October 2016, 03:55
The CSV/OFX/QIF exports from my bank are a full transaction history for the account, all the way back to opening the account. This was okay for a while, but the imports have become *really* slow. We're talking about 30 seconds to process the import of about 2500 transactions (between when I choose which import configuration to use and when it shows me the list of transactions with info and whether they're being imported/reconciled/ignored).
If there's not already one that I've missed, a historical limit of some kind would be pretty useful. e.g. an option along the lines of "don't process transactions older than N days", or "only process the first N transactions" (as part of the import configuration).
This would speed things up a lot, would avoid potential unintentional re-imports of really old data, and simplify the list of transactions for me to check if I want to adjust the auto-matching conditions (right now if I want to change the +/-3 days option to something else, i need to scroll through 2500 transactions to see that none are affected by it).
For the latter issue it seems like it'd be useful to be able to show/hide the 'ignored' transactions in the last step of the import, so that I can quickly check for any oddities (like re-imports of old data).
Regardless of the historical issue, it also seems like the import code could do with some optimisation. I have modern, high-spec computer (quad core 2.6 GHz i7 CPU), it seems very strange to me that it takes 30 seconds to process 2500 entries (even on an old computer it seems like this shouldn't require 30 seconds of processing time!). Perhaps only a few people are processing thousands of transactions at once, so optimisation isn't really warranted, but it strikes me as surprisingly slow nonetheless!
If there's not already one that I've missed, a historical limit of some kind would be pretty useful. e.g. an option along the lines of "don't process transactions older than N days", or "only process the first N transactions" (as part of the import configuration).
This would speed things up a lot, would avoid potential unintentional re-imports of really old data, and simplify the list of transactions for me to check if I want to adjust the auto-matching conditions (right now if I want to change the +/-3 days option to something else, i need to scroll through 2500 transactions to see that none are affected by it).
For the latter issue it seems like it'd be useful to be able to show/hide the 'ignored' transactions in the last step of the import, so that I can quickly check for any oddities (like re-imports of old data).
Regardless of the historical issue, it also seems like the import code could do with some optimisation. I have modern, high-spec computer (quad core 2.6 GHz i7 CPU), it seems very strange to me that it takes 30 seconds to process 2500 entries (even on an old computer it seems like this shouldn't require 30 seconds of processing time!). Perhaps only a few people are processing thousands of transactions at once, so optimisation isn't really warranted, but it strikes me as surprisingly slow nonetheless!
