Open source code. GitHub and other cloud repositories enable developers to share and consume code for almost any purpose imaginable. This reflects today's practical, non-ideological open source culture: Why code it yourself if someone else is offering it free under the most liberal license imaginable?
Cloud APIs. Cloud service APIs from Google, Facebook, LinkedIn, and PayPal have become stalwarts for Web and mobile developers -- and are easily integrated into microservices architectures. Hot new APIs like those offered by Stripe for e-payments emerge all the time, along with more specialized plays such as the popular Twilio for telecom services.
Frameworks everywhere. Programming frameworks, available for all the popular languages, free developers from having to worry about the nonessential details of application development. Choosing the right programming frameworks for the job has become so critical, they've even been referred to as the new programming languages.
Wrapped around these prebuilt elements are modern development approaches such as agile methodology, which stipulates a recursive, piece-by-piece development process that continually solicits feedback from business stakeholders. Devops tools enable developers to provision their own virtual infrastructure -- or, alternatively, have operations reconfigure dev and test environments faster.
Underlying this new, high-speed application assembly line is cloud infrastructure. Wildly unpredictable fluctuations in the number of public users, as well as demands on shared services that may be used by many applications, requires an infrastructure that can pour on compute, storage, or network resources as needed.
For customer-facing applications, cloud has become the default. In most cases, enterprises are turning to public IaaS or PaaS providers such as Amazon Web Services or Microsoft Azure rather than trying to build private clouds from scratch.
The new analytics
Perhaps the most profitable area of big data involves gathering clickstream data about user behavior to optimize applications and make it easier to, say, compare and purchase products through an e-commerce application. Big Web companies such as Yahoo are way ahead in this area, with petabytes of data on HDFS to support mobile, search, advertising, personalization, media, and communications efforts.
Enterprises are pouring money into Hadoop, Spark, and Storm deployments -- as well technologies such as Hive or Impala that enable you to query Hadoop using SQL. The most exciting area today, however, is streaming analytics, where events are processed in near real time rather than in batches -- using clusters of servers packed with huge amounts of memory. The Storm-plus-Kafka combination is emerging as a popular streaming solution, but there are literally dozens of open source projects in the Hadoop ecosystem to experiment with.
Enterprise adoption of these new analytics solutions tends to be somewhat haphazard. Some enterprises encounter problems managing Hadoop at scale; others experiment without clear objectives, resulting in initiatives that never get off the ground. Still others may roll out unconnected projects using similar technology and duplicate their efforts unnecessarily. To avoid the latter case, InfoWorld's Andrew Oliver notes that deploying "Hadoop as a service" is becoming a common pattern: With sufficient preparation, various business units can obtain Hadoop analytics self-service style from a large, centralized, scalable hub.
Sign up for Computerworld eNewsletters.