HBR: 数据科学家是21世纪最性感的职业

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

《哈佛商业评论》2012年10月刊
by Thomas H. Davenport and D.J. Patil                                       

摘要：

有什么能力使数据科学家成功？想想他（她）是数据黑客，分析师，沟通者，和值得信赖的顾问的混合体。组合是非常强大且罕见的。
数据科学家的最基本，最通用的技巧是写代码的能力。这可能在5年之后不太真实，将有更多的人将“科学家”的Titil印在名片上。更经得起时间考验的将是需要数据科学家在同所有利益相关者语言沟通，并证明用数据讲故事的特殊技能，无论是口头，视觉，还是思想上的。
但是，我们要说的是数据科学家中的显性遗传基因，是一种强烈好奇心-透过问题表面之下进行探索的意愿，切中要害发现问题，并提炼成一个很清晰的可以测试的假设。这往往需要在任何领域最具创造性的科学家的联想思维特点。例如，我们知道的数据科学家研究的欺诈问题，他们意识到这是类似于DNA序列问题的类型。通过汇集这些不同的世界，他和他的团队能够创造出一个解决方案，大大降低欺诈损失。

今天的数据科学家类似于华尔街20世纪80年代和90年代的“金融量化工程师”（Quants,俗称“Quan工，Quan客）。在那些日子里物理和数学背景的人流向投资银行和对冲基金，在那里他们可以设计出完全新的算法和数据战略。各种大学开设金融工程硕士课程，生产了第二代的人才，更容易被主流公司接受。这种模式在20世纪90年代的搜索工程师中重复出现，其稀缺技能很快就被计算机科学课程教授。


作者：
Thomas H. Davenport is a visiting professor at Harvard Business School, a senior adviser to Deloitte Analytics, and a coauthor of Judgment Calls (Harvard Business Review Press, 2012). D.J. Patil is the data scientist in residence at Greylock Partners, was formerly the head of data products at LinkedIn, and is the author of Data Jujitsu: The Art of Turning Data into Product (O’Reilly Media, 2012).《数据柔术：将数据转化为产品的艺术》

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏1 回帖

关键词：数据科学家 21世纪 数据科学 HBR 科学家 2011 colleagues 世纪 networking LinkedIn

When Jonathan Goldman arrived for work in June 2006 at LinkedIn, the business networking site, the place still felt like a start-up. The company had just under 8 million accounts, and the number was growing quickly as existing members invited their friends and colleagues to join. But users weren’t seeking out connections with the people who were already on the site at the rate executives had expected. Something was apparently missing in the social experience. As one LinkedIn manager put it, “It was like arriving at a conference reception and realizing you don’t know anyone. So you just stand in the corner sipping your drink—and you probably leave early.”

Goldman, a PhD in physics from Stanford, was intrigued by the linking he did see going on and by the richness of the user profiles. It all made for messy data and unwieldy analysis, but as he began exploring people’s connections, he started to see possibilities. He began forming theories, testing hunches, and finding patterns that allowed him to predict whose networks a given profile would land in. He could imagine that new features capitalizing on the heuristics he was developing might provide value to users. But LinkedIn’s engineering team, caught up in the challenges of scaling up the site, seemed uninterested. Some colleagues were openly dismissive of Goldman’s ideas. Why would users need LinkedIn to figure out their networks for them? The site already had an address book importer that could pull in all a member’s connections.

Luckily, Reid Hoffman, LinkedIn’s cofounder and CEO at the time (now its executive chairman), had faith in the power of analytics because of his experiences at PayPal, and he had granted Goldman a high degree of autonomy. For one thing, he had given Goldman a way to circumvent the traditional product release cycle by publishing small modules in the form of ads on the site’s most popular pages.

Through one such module, Goldman started to test what would happen if you presented users with names of people they hadn’t yet connected with but seemed likely to know—for example, people who had shared their tenures at schools and workplaces. He did this by ginning up a custom ad that displayed the three best new matches for each user based on the background entered in his or her LinkedIn profile. Within days it was obvious that something remarkable was taking place. The click-through rate on those ads was the highest ever seen. Goldman continued to refine how the suggestions were generated, incorporating networking ideas such as “triangle closing”—the notion that if you know Larry and Sue, there’s a good chance that Larry and Sue know each other. Goldman and his team also got the action required to respond to a suggestion down to one click.

It didn’t take long for LinkedIn’s top managers to recognize a good idea and make it a standard feature. That’s when things really took off. “People You May Know” ads achieved a click-through rate 30% higher than the rate obtained by other prompts to visit more pages on the site. They generated millions of new page views. Thanks to this one feature, LinkedIn’s growth trajectory shifted significantly upward.
A New Breed

Goldman is a good example of a new key player in organizations: the “data scientist.” It’s a high-ranking professional with the training and curiosity to make discoveries in the world of big data. The title has been around for only a few years. (It was coined in 2008 by one of us, D.J. Patil, and Jeff Hammerbacher, then the respective leads of data and analytics efforts at LinkedIn and Facebook.) But thousands of data scientists are already working at both start-ups and well-established companies. Their sudden appearance on the business scene reflects the fact that companies are now wrestling with information that comes in varieties and volumes never encountered before. If your organization stores multiple petabytes of data, if the information most critical to your business resides in forms other than rows and columns of numbers, or if answering your biggest question would involve a “mashup” of several analytical efforts, you’ve got a big data opportunity.

Much of the current enthusiasm for big data focuses on technologies that make taming it possible, including Hadoop (the most widely used framework for distributed file system processing) and related open-source tools, cloud computing, and data visualization. While those are important breakthroughs, at least as important are the people with the skill set (and the mind-set) to put them to good use. On this front, demand has raced ahead of supply. Indeed, the shortage of data scientists is becoming a serious constraint in some sectors. Greylock Partners, an early-stage venture firm that has backed companies such as Facebook, LinkedIn, Palo Alto Networks, and Workday, is worried enough about the tight labor pool that it has built its own specialized recruiting team to channel talent to businesses in its portfolio. “Once they have data,” says Dan Portillo, who leads that team, “they really need people who can manage it and find insights in it.”

数据科学家是什么样的人？

Who Are These People?

If capitalizing on big data depends on hiring scarce data scientists, then the challenge for managers is to learn how to identify that talent, attract it to an enterprise, and make it productive. None of those tasks is as straightforward as it is with other, established organizational roles. Start with the fact that there are no university programs offering degrees in data science. There is also little consensus on where the role fits in an organization, how data scientists can add the most value, and how their performance should be measured.

The first step in filling the need for data scientists, therefore, is to understand what they do in businesses. Then ask, What skills do they need? And what fields are those skills most readily found in?

More than anything, what data scientists do is make discoveries while swimming in data. It’s their preferred method of navigating the world around them. At ease in the digital realm, they are able to bring structure to large quantities of formless data and make analysis possible. They identify rich data sources, join them with other, potentially incomplete data sources, and clean the resulting set. In a competitive landscape where challenges keep changing and data never stop flowing, data scientists help decision makers shift from ad hoc analysis to an ongoing conversation with data.

Data scientists realize that they face technical limitations, but they don’t allow that to bog down their search for novel solutions. As they make discoveries, they communicate what they’ve learned and suggest its implications for new business directions. Often they are creative in displaying information visually and making the patterns they find clear and compelling. They advise executives and product managers on the implications of the data for products, processes, and decisions.

Given the nascent state of their trade, it often falls to data scientists to fashion their own tools and even conduct academic-style research. Yahoo, one of the firms that employed a group of data scientists early on, was instrumental in developing Hadoop. Facebook’s data team created the language Hive for programming Hadoop projects. Many other data scientists, especially at data-driven companies such as Google, Amazon, Microsoft, Walmart, eBay, LinkedIn, and Twitter, have added to and refined the tool kit.

What kind of person does all this? What abilities make a data scientist successful? Think of him or her as a hybrid of data hacker, analyst, communicator, and trusted adviser. The combination is extremely powerful—and rare.

Data scientists’ most basic, universal skill is the ability to write code. This may be less true in five years’ time, when many more people will have the title “data scientist” on their business cards. More enduring will be the need for data scientists to communicate in language that all their stakeholders understand—and to demonstrate the special skills involved in storytelling with data, whether verbally, visually, or—ideally—both.

But we would say the dominant trait among data scientists is an intense curiosity—a desire to go beneath the surface of a problem, find the questions at its heart, and distill them into a very clear set of hypotheses that can be tested. This often entails the associative thinking that characterizes the most creative scientists in any field. For example, we know of a data scientist studying a fraud problem who realized that it was analogous to a type of DNA sequencing problem. By bringing together those disparate worlds, he and his team were able to craft a solution that dramatically reduced fraud losses.

Perhaps it’s becoming clear why the word “scientist” fits this emerging role. Experimental physicists, for example, also have to design equipment, gather data, conduct multiple experiments, and communicate their results. Thus, companies looking for people who can work with complex data have had good luck recruiting among those with educational and work backgrounds in the physical or social sciences. Some of the best and brightest data scientists are PhDs in esoteric fields like ecology and systems biology. George Roumeliotis, the head of a data science team at Intuit in Silicon Valley, holds a doctorate in astrophysics. A little less surprisingly, many of the data scientists working in business today were formally trained in computer science, math, or economics. They can emerge from any field that has a strong data and computational focus.

It’s important to keep that image of the scientist in mind—because the word “data” might easily send a search for talent down the wrong path. As Portillo told us, “The traditional backgrounds of people you saw 10 to 15 years ago just don’t cut it these days.” A quantitative analyst can be great at analyzing data but not at subduing a mass of unstructured data and getting it into a form in which it can be analyzed. A data management expert might be great at generating and organizing data in structured form but not at turning unstructured data into structured data—and also not at actually analyzing the data. And while people without strong social skills might thrive in traditional data professions, data scientists must have such skills to be effective.

如何发掘你所需的数据科学家？
How to Find the Data Scientists You Need

Roumeliotis was clear with us that he doesn’t hire on the basis of statistical or analytical capabilities. He begins his search for data scientists by asking candidates if they can develop prototypes in a mainstream programming language such as Java. Roumeliotis seeks both a skill set—a solid foundation in math, statistics, probability, and computer science—and certain habits of mind. He wants people with a feel for business issues and empathy for customers. Then, he says, he builds on all that with on-the-job training and an occasional course in a particular technology.

Several universities are planning to launch data science programs, and existing programs in analytics, such as the Master of Science in Analytics program at North Carolina State, are busy adding big data exercises and coursework. Some companies are also trying to develop their own data scientists. After acquiring the big data firm Greenplum, EMC decided that the availability of data scientists would be a gating factor in its own—and customers’—exploitation of big data. So its Education Services division launched a data science and big data analytics training and certification program. EMC makes the program available to both employees and customers, and some of its graduates are already working on internal big data initiatives.

As educational offerings proliferate, the pipeline of talent should expand. Vendors of big data technologies are also working to make them easier to use. In the meantime one data scientist has come up with a creative approach to closing the gap. The Insight Data Science Fellows Program, a postdoctoral fellowship designed by Jake Klamka (a high-energy physicist by training), takes scientists from academia and in six weeks prepares them to succeed as data scientists. The program combines mentoring by data experts from local companies (such as Facebook, Twitter, Google, and LinkedIn) with exposure to actual big data challenges. Originally aiming for 10 fellows, Klamka wound up accepting 30, from an applicant pool numbering more than 200. More organizations are now lining up to participate. “The demand from companies has been phenomenal,” Klamka told us. “They just can’t get this kind of high-quality talent.”

为何数据科学家愿意在这里工作？
Why Would a Data Scientist Want to Work Here?

Even as the ranks of data scientists swell, competition for top talent will remain fierce. Expect candidates to size up employment opportunities on the basis of how interesting the big data challenges are. As one of them commented, “If we wanted to work with structured data, we’d be on Wall Street.” Given that today’s most qualified prospects come from nonbusiness backgrounds, hiring managers may need to figure out how to paint an exciting picture of the potential for breakthroughs that their problems offer.

Pay will of course be a factor. A good data scientist will have many doors open to him or her, and salaries will be bid upward. Several data scientists working at start-ups commented that they’d demanded and got large stock option packages. Even for someone accepting a position for other reasons, compensation signals a level of respect and the value the role is expected to add to the business. But our informal survey of the priorities of data scientists revealed something more fundamentally important. They want to be “on the bridge.” The reference is to the 1960s television show Star Trek, in which the starship captain James Kirk relies heavily on data supplied by Mr. Spock. Data scientists want to be in the thick of a developing situation, with real-time awareness of the evolving set of choices it presents.

Considering the difficulty of finding and keeping data scientists, one would think that a good strategy would involve hiring them as consultants. Most consulting firms have yet to assemble many of them. Even the largest firms, such as Accenture, Deloitte, and IBM Global Services, are in the early stages of leading big data projects for their clients. The skills of the data scientists they do have on staff are mainly being applied to more-conventional quantitative analysis problems. Offshore analytics services firms, such as Mu Sigma, might be the ones to make the first major inroads with data scientists.

But the data scientists we’ve spoken with say they want to build things, not just give advice to a decision maker. One described being a consultant as “the dead zone—all you get to do is tell someone else what the analyses say they should do.” By creating solutions that work, they can have more impact and leave their marks as pioneers of their profession.

关照与培养
Care and Feeding

Data scientists don’t do well on a short leash. They should have the freedom to experiment and explore possibilities. That said, they need close relationships with the rest of the business. The most important ties for them to forge are with executives in charge of products and services rather than with people overseeing business functions. As the story of Jonathan Goldman illustrates, their greatest opportunity to add value is not in creating reports or presentations for senior executives but in innovating with customer-facing products and processes.

LinkedIn isn’t the only company to use data scientists to generate ideas for products, features, and value-adding services. At Intuit data scientists are asked to develop insights for small-business customers and consumers and report to a new senior vice president of big data, social design, and marketing. GE is already using data science to optimize the service contracts and maintenance intervals for industrial products. Google, of course, uses data scientists to refine its core search and ad-serving algorithms. Zynga uses data scientists to optimize the game experience for both long-term engagement and revenue. Netflix created the well-known Netflix Prize, given to the data science team that developed the best way to improve the company’s movie recommendation system. The test-preparation firm Kaplan uses its data scientists to uncover effective learning strategies.

There is, however, a potential downside to having people with sophisticated skills in a fast-evolving field spend their time among general management colleagues. They’ll have less interaction with similar specialists, which they need to keep their skills sharp and their tool kit state-of-the-art. Data scientists have to connect with communities of practice, either within large firms or externally. New conferences and informal associations are springing up to support collaboration and technology sharing, and companies should encourage scientists to become involved in them with the understanding that “more water in the harbor floats all boats.”

Data scientists tend to be more motivated, too, when more is expected of them. The challenges of accessing and structuring big data sometimes leave little time or energy for sophisticated analytics involving prediction or optimization. Yet if executives make it clear that simple reports are not enough, data scientists will devote more effort to advanced analytics. Big data shouldn’t equal “small math.”

未来十年最热门职位
The Hot Job of the Decade

Hal Varian, the chief economist at Google, is known to have said, “The sexy job in the next 10 years will be statisticians. People think I’m joking, but who would’ve guessed that computer engineers would’ve been the sexy job of the 1990s?”

If “sexy” means having rare qualities that are much in demand, data scientists are already there. They are difficult and expensive to hire and, given the very competitive market for their services, difficult to retain. There simply aren’t a lot of people with their combination of scientific background and computational and analytical skills.

Data scientists today are akin to Wall Street “quants” of the 1980s and 1990s. In those days people with backgrounds in physics and math streamed to investment banks and hedge funds, where they could devise entirely new algorithms and data strategies. Then a variety of universities developed master’s programs in financial engineering, which churned out a second generation of talent that was more accessible to mainstream firms. The pattern was repeated later in the 1990s with search engineers, whose rarefied skills soon came to be taught in computer science programs.

One question raised by this is whether some firms would be wise to wait until that second generation of data scientists emerges, and the candidates are more numerous, less expensive, and easier to vet and assimilate in a business setting. Why not leave the trouble of hunting down and domesticating exotic talent to the big data start-ups and to firms like GE and Walmart, whose aggressive strategies require them to be at the forefront?

The problem with that reasoning is that the advance of big data shows no signs of slowing. If companies sit out this trend’s early days for lack of talent, they risk falling behind as competitors and channel partners gain nearly unassailable advantages. Think of big data as an epic wave gathering now, starting to crest. If you want to catch it, you need people who can surf.

2006年Jonathan Goldman到商业社交网站LinkedIn工作，那时的LinkedIn还只是刚创业不久，网站注册人数不到8百万，但是很多成员会邀请自己的朋友和同学加入，因此注册人数迅速增加。但是用户要找到已经在网站注册的用户不太容易，比例达不到管理人员的期望值。很明显，有些社交体验缺失了。如同一位LinkedIn管理者说的，“这就好像，你到了会议接待处，结果发现一个人都不认识，你只好站到一边，一个人小酌­——很可能你早早地离开了。”

Goldman是斯坦福物理学博士毕业，他非常着迷于越来越多的用户关联和丰富的用户个人资料。这些原本只能带来一堆杂乱的数据和笨拙的分析，但是，当他开始探究用户之间的联系时，他开始看到新的可能。于是他开始组织他的理论，检验他的猜想，建立模型，预测用户愿意与谁建立联系。他感到，他正在开发的新功能，能带给用户价值。但是LinkedIn的工程师们当时忙于提升网站性能，没有理睬，有些同事则公开表示不看好Goldman的想法：为什么用户想要LinkedIn告诉他们该和哪些用户建立联系呢？网站已经有一个导入通讯录的功能，能导入用户的所有联系人。

幸运的是，公司的联合创世人兼当时的CEO Reid Hoffman根据自己在PayPal的经验，相信数据分析的强大力量，给予了Goldman高度的自主权。其中一项就是，Goldman可以绕开传统的产品发布流程，而以广告的形式把这个小模块发布在网站最受欢迎的页面上。

通过这个模块，Goldman开始了他的试验，用户可能认识一些人，比如和用户来自同一个学校或工作单位, 却还没有在网站上建立起这些关系，如果把这些名字告诉用户，他们会作何反应。他根据用户在网站上注册时填写的背景资料，找出了每个用户可能最想与之建立联系的三个用户，然后定制了一套广告。几天之内，很明显地，奇妙的事情发生了，这些广告的点击率前所未有的高。接下来，Goldman根据“闭环理论”改进了他的推荐方法，闭环理论指的是如果你同时认识张三和李四，那么张三和李四很可能也相互认识。同时，Goldman和他的团队让用户对每个推荐的操作可以一键搞定。

很快LinkedIn的高层开始认识到这是个很好的主意，并将其列为标准功能。从那时起，事情真的开始起飞了。“你可能认识的人（People You May Know）”广告获得了30%的点击率，比其他任何的站内推广广告点击率都要高，共计产生了数百万个新页面浏览。得益于这项新功能，LinkedIn的成长速度大幅提升。

新新职业

Goldman 是一个很好的例子来说明组织中的重要新成员——“数据科学家”。这是非常高阶的专业岗位，要有在数据海洋中寻宝的好奇心和相应训练。这个头衔存在有几年了，第一次出现是2008由 D.J. Patil(本文作者之一)和Jeff Hammerbacher提出的，他们后来分别成为了LinkedIn和Facebook的数据和分析团队的负责人。但现在已经有数千位数据科学家工作于创业公司和成熟的大型企业。他们在行业里的忽然走俏，反应了这样一个现状，企业需要处理的信息正以从未遇见过的规模和渠道涌现。如果你的机构存储了几个PB的数据，或者对于你的生意最重要的信息是表格式的，而不再是行列的数据，或者要回答你最大的问题需要各种分析手段的“混搭”，你赶上大数据时代了。

现阶段对于大数据的主要热情都集中在大数据的处理技术上，比如，使用最广泛的分布式文件处理系统Hadoop，和相关的开源工具、云计算、数据可视化技术。这些突破性技术都是非常重要的，重要程度就不亚于有能力与脑力运用好技术的人。对数据科学家的需求快速增加，已经超过了供给，事实上，人才缺乏开始严重制约某些行业。Greylock Partners是一家投资初创企业的风投公司，曾经投资过Facebook, LinkedIn, Palo Alto Networks和Workday，它非常担忧紧张的人才储备，因而建立了自己的招聘团队，负责给自己投资的公司输送人才。招聘团队的负责人Dan Portillo说，“这些公司一旦有了数据，就需要有人管理数据，发现真知。”

他们是谁？

从大数据中获利需要雇佣稀缺的数据科学家，管理人员面临三大挑战，识别人才，吸引人才，善用人才。和其他职责明确的岗位相比，这三项任务都不那么直接明了。首先，目前没有高校项目培养相关人才，同时，数据科学家在组织中处于什么位置，如何让他们创造最大价值，如何衡量他们的作用，这些都没有公认的标准。

因此，要想挖掘出数据科学家，首先要明白他们在业务中能干什么，其次，他们需要哪些技能？哪些现有的领域会用到这些技能？

数据科学家首要任务是在数据的海洋中探索发现，他们更喜欢用这种方式看待周围的世界。他们要在数字王国里游刃有余，把大量散乱的数据变成结构化的可供分析的数据，还要找出丰富的数据源，整合其他可能不完整的数据源，并清理成结果数据集。新的竞争环境中，挑战不断地变化，新数据不断地流入，数据科学家需要帮助决策者穿梭于各种分析，从临时数据分析(ad hoc)到持续的数据交互分析。

数据科学家会遇到技术的局限性，但不会让技术阻扰他们寻找新颖的解决方案。当他们有所发现，便交流他们的发现，建议新的业务方向。通常他们很有创造力的展示视觉化的信息，也让找到的模式清晰而有说服力。他们会把蕴含在数据中的规律建议给产品经理和主管们，从而影响产品，流程，和决策。

由于这中行当还处于初级阶段，数据科学家常常会推广他们自己开发的工具，甚至进行学术研究。雅虎之前雇佣的一批数据科学家开发出了Hadoop。Facebook的数据团队开发了在Hadoop上编程的Hive语言。很多其他的数据科学家都丰富或者优化了这套工具，尤其是数据驱动的公司，比如谷歌，亚马逊，微软，沃尔玛，eBay，LinkedIn, 和twitter。

什么样的人有能力做这些呢？什么技能让数据科学家成功呢？你可以把他们看成是数据骇客，分析师，沟通高手，值得信任的咨询师，这些东西组合到一起极具威力，也极其少见。

数据科学家最基本最通用的技能是写代码。也许五年后不太会这样了，那时很多人都会在他们的名片上印着“数据科学家”。一个更保值的技能是用所有相关方面都能听得懂语言进行沟通，另一个是用数据讲故事的特殊能力，通过口头表达或者视觉效果，或者两者都有。

但我们觉得，数据科学家占支配地位的品质应该是强烈的好奇心，想要深入问题内部的渴望，找到最核心的问题，提取成清晰的结论，并要经得起检验。比如，我们所知道的一位数据科学家，他研究的是欺诈问题，但他发现这个问题和DNA排序问题非常类似，在融合了两个完全不相干的世界之后，他和他的团队找到了一种能大幅降低欺诈损失的解决方案。

现在你大概清楚了为什么这个新兴的角色会被称为“科学家”。比如实验物理学家，同样也需要设计仪器，收集数据，反复试验，并最终展示结果。因此，很多公司寻找能处理复杂数据的人才，可很多招到的不错的人才都是有物理或社会科学领域的学习和工作背景。有些最好的最有前途的数据科学家是研究复杂科学的博士生，比如生态学或者系统生物学。George是硅谷Intuit公司的数据科学团队的负责人，本身是天文学博士毕业。更普遍的是，当今业界许多数据科学家毕业于计算机科学，数学，经济学，和任何数据和计算密集型的领域。