Spark内核设计的艺术：架构设计与实现pdf

相似文件 换一批

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

本书赞誉
前言
第1章　环境准备 ········································1
1.1　运行环境准备 ···········································2
1.1.1　安装JDK ·········································2
1.1.2　安装Scala ········································2
1.1.3　安装Spark ·······································3
1.2　Spark初体验 ···································4
1.2.1　运行spark-shell ·······························4
1.2.2　执行word count ······························5
1.2.3　剖析spark-shell ·······························9
1.3　阅读环境准备 ·········································14
1.3.1　安装SBT ·······································15
1.3.2　安装Git ·········································15
1.3.3　安装Eclipse Scala IDE插件 ········15
1.4　Spark源码编译与调试 ·························17
1.5　小结 ···························23
第2章　设计理念与基本架构 ···············24
2.1　初识Spark ··································25
2.1.1　Hadoop MRv1的局限···················25
2.1.2　Spark的特点 ·································26
2.1.3 Spark使用场景 ·····························28
2.2　Spark基础知识 ······································29
2.3　Spark基本设计思想 ·····························31
2.3.1　Spark模块设计 ·····························32
2.3.2　Spark模型设计 ·····························34
2.4　Spark基本架构 ···································36
2.5　小结 ·································38
第3章　Spark基础设施 ·························39
3.1　Spark配置 ········································40
3.1.1　系统属性中的配置 ·······················40
3.1.2　使用SparkConf配置的API ·········41
3.1.3　克隆SparkConf配置 ····················42
3.2　Spark内置RPC框架 ····························42
3.2.1　RPC配置TransportConf ··············45
3.2.2　RPC客户端工厂Transport- ClientFactory ·······················47
3.2.3　RPC服务端TransportServer ········53
3.2.4　管道初始化 ···································56
3.2.5　TransportChannelHandler详解 ·····57
3.2.6　服务端RpcHandler详解 ··············63
3.2.7　服务端引导程序Transport-ServerBootstrap ·····················68
3.2.8　客户端TransportClient详解 ········71
3.3　事件总线 ····································78
3.3.1　ListenerBus的继承体系 ···············79
3.3.2　SparkListenerBus详解 ··················80
3.3.3　LiveListenerBus详解 ····················83
3.4　度量系统 ···········································87
3.4.1　Source继承体系 ···························87
3.4.2　Sink继承体系 ·······························89
3.5　小结 ·········································92
第4章　SparkContext的初始化 ·········93
4.1　SparkContext概述 ·································94
4.2　创建Spark环境 ·····································97
4.3　SparkUI的实现 ····································100
4.3.1　SparkUI概述 ·······························100
...

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏1 回帖