Program Optimization
From Federal Burro of Information
Beta rules of program optimization: run the program many time and watch closely
- time it - watch it's IO / CPU / memory usage. - watch it's file open()s. - have tea.
Also: Write down in English what it is you are trying to do. Is it doing what you think it's doing?
- lots of data lots of reports
- lots of data one report?
- lots of reports one dataset?
- batchable? threadable? increamental? can parts of the report be run on their own and then added together at the end?
- Do you have to do all the reporting at the same time ? Could you do some of the data every day such that at the end of the month all the * required reports are done? Rolling reports?
- How long should it take? and a year from now how long should it take?
- What is your budget for computing resources? and a year from now what sort of budget will you have?