作者：George Hill ，他是知名商業媒體 Innovation Enterprise 的主編，同時也是 The Cyclist 公司的聯合創始人。本文由可譯網toypipi , 中山狼 , 薯片番茄, 班納睿翻譯。

長期以來，Hadoop 這個詞鋪天蓋地，幾乎成了大數據的代名詞。三年之前，提起超越 Hadoop 這件事，似乎還顯得難以想象。但三年后的今天，這一情況發生了一些改變。

[[184783]]

早在 2012 年，知名媒體 SiliconANGLE 就針對 Twitter 平臺上的大數據專業人士做了一項調查。調查結果顯示：這些專業人士日常談論 NoSQL 等技術（如 MongoDB）的次數要遠多于 Hadoop。這表明，至少在數據科學家的群體中，用 Hadoop 代指大數據似乎并不準確

然而大多數人認為 Hadoop 已經是大數據最重要的技術之一，是大數據構建的基礎。它還被利用在一些新的領域，如倉儲系統。話雖如此，出人意料的是，它的適用性或多或少有點滯后。對此，IBM Software 的傳道士 James Kobielus 說道：“ 2016 年，Hadoop 在大數據領域的下滑速度比我預期的要快得多。”

其中原因很難說清，但可將其理解為數據領域的慣有現象。Gartner 于 2015 年的調查顯示，54% 的公司都沒有計劃投資 Hadoop，另外 44% 的公司表示已使用 Hadoop 或將在未來兩年使用。這些數據不同人看來有不同的觀點，你可以認為 Hadoop 將進一步擴大，也可以認為大多數人根本不重視 Hadoop。同時，調查還揭露了一些其他無法平息的影響因素。在沒有投資的人當中，49% 的人仍在努力挖掘 Hadoop 的使用價值，而另外 57% 的人指出，其中的技能差距是決定是否使用的主要阻礙，而這并不能立馬得到解決。這一現象恰好與 ”Hadoop Testing“ 關于就業趨勢的調查結果相一致：在 2014 年中旬，這一關鍵詞在大約 0.061% 的廣告中出現，在 2016 年末又增長至 0.087%，在 18 個月內，增長了約 43%。

這可能表明，采用Hadoop的公司數量不一定會降低到坊間證據表明的那樣，但公司只是發現很難從他們現有的團隊中提取Hadoop的價值，他們需要更多的專業知識。

另一個可能引起人們關注的因素是，一個人的大數據卻是另一個人的小數據。 Hadoop是為大量數據而設計的，Kashif Saiyed在KD Nuggets上寫道：‘如果你的企業沒有真正面臨海量數據的問題，你就不需要Hadoop，因此數百家企業對他們無用的、處理2到10TB數據規模大小的 Hadoop集群感到非常失望 – Hadoop技術只是不擅長處理這種規模。‘

大多數公司目前沒有足夠的數據來保證Hadoop的部署，但還是這么做的原因是他們覺得他們需要互相攀比。 經過幾年的實驗，并與真正的數據科學家一起工作，他們很快就意識到他們的數據在其他技術上工作得更好。

這種趨勢已經超出了采用開源平臺的速度，但對于一些公司來說，這已經產生了實際的財務影響。 Cloudera和Hortonworks是從Hadoop框架構建自己產品的兩家最大的公司。 由于Hadoop的下滑，對于兩家公司都造成了不同程度的重大損失，據報告Cloudera失去了40％，而Hortonworks的股價自2015年中期以來已經下跌了68％。

這篇文章對Hadoop的批評似乎有些苛刻，但并不是平臺本身造成了當前的問題。 相反，這可能是由于過分炒作和大數據協會導致了事實上的傷害。一些公司采用了該平臺卻沒有理解它，同時又沒有合適的人或數據來使其正常工作，這導致了項目實施的幻滅和明顯的停滯。Hadoop依然還有強大的生命力，只是人們需要更好地理解它。

原文：

Three years ago, looking beyond Hadoop was insanity, and there was little else that could come close according to many in the media. However, the reality has been a little different.

For a long period, Hadoop and big data were almost interchangeable when they were being discussed by those in the media, although this was not necessarily found to be the case amongst data scientists. A study by Silicon Angle in 2012 analyzing Twitter conversations between data professionals talking about big data found that they actually talked about NoSQL technologies like MongoDB as much, or more, than Hadoop, which would indicate that it has not actually been the must have that many assumed it was.

Most would argue that Hadoop has been one of the single most important elements in the spread of big data, that it is very much the foundation on which data today is built. We are also still finding new ways to use it, in warehousing for instance. That being said, to the surprise of many, its adoption appears to have more or less stagnated, leading even James Kobielus, Big Data Evangelist at IBM Software, to claim that ‘Hadoop declined more rapidly in 2016 from the big-data landscape than I expected.’

The reasons for this are hard to ascertain, but could be down to a problem common in data circles. A 2015 study from Gartner found that 54% of companies had no plans to invest in Hadoop, while 44% of those asked had adopted Hadoop already or planned to at some point in the next two years. This could, depending on your point of view, be taken to mean either that it would see even further expansion or that the majority were ignoring it. However, the survey also revealed a number of other telling factors with implications unlikely to have subsided since. Of those who were not investing, 49% were still trying to figure out how to use it for value, while 57% said that the skills gap was the major reason, a number that is not going to be corrected overnight. This coincides with findings from Indeed who tracked job trends with ‘Hadoop Testing’ in the title, with the term featured in a peak of 0.061% of ads in mid 2014, which then jumped to 0.087% in late 2016, an increase of around 43% in 18 months.

What this may signal is that adoption hasn’t necessarily dropped to the extent that anecdotal evidence would suggest, but companies are simply finding it difficult to extract value from Hadoop from their current teams and they require greater expertise.

Another element that may be cause for concern is simply that one man’s big data is another man’s small data. Hadoop is designed for huge amounts of data, and as Kashif Saiyed wrote on KD Nuggets ‘You don’t need Hadoop if you don’t really have a problem of huge data volumes in your enterprise, so hundreds of enterprises were hugely disappointed by their useless 2 to 10TB Hadoop clusters – Hadoop technology just doesn’t shine at this scale.’

Most companies do not currently have enough data to warrant a Hadoop rollout, but did so anyway because they felt they needed to keep up with the Joneses. After a few years of experimentation and working alongside genuine data scientists, they soon realize that their data works better in other technologies.

This trend has had impacts beyond a slow down in the adoption of an open source platform though, for some companies this has had real world financial impacts. Cloudera and Hortonworks are two of the biggest companies that build their products out from a Hadoop framework. Both have lost significant value in-part due to its decline, with Cloudera reported to have lost 40% whilst Hortonworks’ shares have plummeted 68% since mid 2015.

Criticism within this article may seem harsh on Hadoop, but it is not the platform in itself that has caused the current issues. Instead it is perhaps the hype and association of big data that has done the real damage. Companies have adopted the platform without understanding it and then failed to get the right people or data to make it work properly, which has led to disillusionment and its apparent stagnation. There is still a huge amount of life in Hadoop, but people just need to understand it better.