I am testing in a complex, integrated environment that includes Peoplesoft. The current methodology is to take a copy of the entire system's data to use as the QA test bed. This is cumbersome. Refreshes can takes days. I don't need that much data. (I am not responsible for load testing.) Are you aware of any tools or methodologies I could leverage to take partial "snapshots" of the data? My goal would be to know exactly what data I have in the testing environment and then to be able to do painless refreshes.
Requires Free Membership to View
When you register, you'll receive targeted emails designed to keep you informed of the most relevant information on Agile development, application security, testing & QA, software requirements, and more.
Hannah Smalltree, Editorial Director
|
 |
I've worked in testing environments where I've had partial snapshots (of production data) and also environments with a full production load. I've worked in environments where I've had no data and have created the database using a "create script" constructed by the database administrator, installed software. and tested with an empty/no data environment. In situations with no data to begin with, I've used the data that gets created manually over time and data that gets added by test automation.
In cases, where I've worked with partial snapshots, the database administrator and/or the system architect has assessed how much data would be practical to load; practical meaning they have determined the amount of data to load based on the limitations of the physical space available or the class of the server or any other aspect of the configuration for which a limit will be encountered. I've worked with DBAs to help define the data set once they've determined the physical limitations. For example, if I understand I can load only X number of records, then I've clarified what type of data I'm looking for such as last month's product inventory or X number of days worth of transactions. It is possible for DBAs to help with this. If you're getting a full data dump and losing time while waiting for the data to load, there are a couple of options:
- Clarify what you need.
- Speak up about down time
- Confirm the refresh process
It might be that there is an assumption that the test environment should have a full copy of production. If customer support uses the test environment when production issues arise, this need might be the cause of why so much data is being loaded.
It might be that your project stakeholders are not aware that you and your team lose time due to the refreshes. Communicate about the lost time. Track the lost time so you're better able to explain the impact.
It might be the case that transaction logging at the database level has been turned on, which can make the loading process a slow one. Check with your DBA to see if logging has been turned off and ask if there are faster operations such as bulk inserts to help reduce down time. Ask if there are other ways to improve the loading time.
I've not had experience with tools in this category, but I have had experience hashing through the refresh process with DBAs and system architects. It could be a case that they're executing the simplest procedure for them to refresh your environment without realizing the lost time on your end. Much like you sometimes need to inspire a developer to repair a bug, you might need to help encourage your DBA to help you and your team save time.
This was first published in October 2008