Any production-ready big data platform needs all of these capabilities. So again, it’s a matter of weighing customization against flexibility. If you go DIY, you should expect to need a significant integration effort. But you will end up with a solution built specifically for your existing security, authentication and policy infrastructure.
If you go with an out-of-the-box solution, you’re getting a platform that’s built from the ground up to meet enterprise security and privacy requirements, including policy-based access control, encryption and auditing out of the box. Some can even dynamically generate different views of data for different users, such as presenting full views of records to some users and de-identified versions to others, on the fly. Just know that you may have to adapt some of your internal processes around a prebuilt platform.
Ultimately, the big data path you choose comes down to understanding your organization. Maybe you have unique requirements that demand a custom solution. Maybe you’re addressing a limited set of questions, or have existing data collection processes and infrastructure that you don’t want to change. If so, a custom-built big data platform tailored precisely to your needs may be the best fit. But if that’s not the case—if big data is just one tool to support your core business strategy—a pre-integrated, enterprise-ready solution can offer a relatively fast, easy way to start unlocking the value of your data.
Weeks drives the technical direction of PHEMI’s big data warehouse product architecture. Prior to joining PHEMI, Russ was a senior software designer in the network management groups at Ericsson and Cray Supercomputers.
Sign up for Computerworld eNewsletters.