Does Agile Work for Data Warehousing?

Agile Data Warehouse / Agile Business Intelligence

Watson Analytics

At Bow River Solutions we have used agile methodologies for many of our business analytics customers in Calgary, Edmonton and Saskatoon and across Western Canada. We have successfully delivered data warehouses and business intelligence systems using agile project management and have been witness to a variety of implementations. The big key for successfully delivering projects using agile is in the full team involvement, specifically having active stakeholders.

Rough Start

When Agile methodologies first started being applied to data warehousing and business intelligence it was a bit of a piecemeal approach of trying to make a system designed for programming fit for IT projects that typically spanned the better part of a year or multiple years. It has been worth the effort to make it work but it definitely was not a perfect fit initially.

Biggest Roadblock

The biggest holdup in trying to make Agile fit with data warehousing was in being able to show something of value in a short period of time like a typical 2 week iteration. This required a different vision of what was of value – where sometimes a new table was of value and had to be displayed during an iteration. But also this required a different approach to the whole project where reporting was not tacked on the end but added in right from the start so stakeholders could see the end result right from the beginning – even if that iteration simply delivered one dimension into the data warehouse and right through to the reporting interface for stakeholders to see, touch and feel.

Biggest Failure

Delivering value in a short iteration was probably the biggest holdup to agile methodologies but the biggest failure was in lack of stakeholder involvement. We have had projects that have tried to go with an agile approach and did succeed in the short iterations. These projects succeeded in tracking their overall progress and could quantitatively show the success of the approach. Problem was they did this all without a highly involved stakeholder. These projects can still succeed but the overall team moral tends to be low. There’s that feeling of “it doesn’t really matter”. When an involved stakeholder is present at scrum meetings and is actively waiting for the incremental deliverable at the end of the iteration there is an excitement that really can’t be built in any other way. This is when the real success of agile data warehousing and business intelligence can be seen and felt and everyone has a vested interest in the project success.

Agile Success

Agile methodologies have brought significant value to big data warehouse and business intelligence projects – changing the delivery significantly, and for the better. The key is to have those active stakeholders available from the start and throughout the project to achieve real success.

Ask us about what we like best about agile projects and how we can help you achieve data warehousing and business intelligence success. Give us a call  (587)-885-1090 or send us an email info@bowriversolutions.com and we can set up a meeting to review to discuss. We can meet locally in our Calgary office at 264 Midpark Way SE, in your office or via a conference call.