���] �p�ɌW(xu��)��(sh��)��(j��)�ھ�:�㷨����(ch��ng)���c��(sh��)��(j��)�a(ch��n)Ʒ

��1�� ��(sh��)��(j��)���ƪ 1 1.1 ��(sh��)��(j��)֮�U 1 1.2 ��(sh��)��(j��)��� 1 1.2.1 ��(sh��)��(j��)����@�Ʉ� 2 1.2.2 ��(du��)��(sh��)��(j��)��ѵ����� 2 1.3 ��(sh��)��(j��)�r(sh��)�����҂� 4 1.4 �ɞ�DT�r(sh��)�������(q��)�� 6 1.4.1 ��(sh��)��(j��)�]�к��� 6 1.4.2 ��(sh��)��(j��)���B(t��i)���} 7 1.4.3 �����Ĕ�(sh��)��(j��)���B(t��i) 8 1.4.4 �Y(ji��)β 8 ��2�� ��(sh��)��(j��)���T 9 2.1 ��������SQL�Ļ��A(ch��)�Z�� 9 2.1.1 ���R(sh��)SQL 9 2.1.2 �W(xu��)��(hu��)����h(hu��n)�� 10 2.1.3 ���õ�SQL�Z������ƪ�� 13 2.1.4 ���õ�SQL�Z������ƪ�� 17 2.2 ��Windows 7����ϵ�y(t��ng)�ϴIPython Notebook 25 2.2.1 �W(xu��)��(x��)Python�ij��� 25 2.2.2 �IPython Notebook 26 2.2.3 IPython.exe Notebook��ʹ���f�� 27 2.2.4 ����IPython Notebook�h(yu��n)���{(di��o)�� 27 2.3 ��������Python�Ļ����Z�� 30 2.4 ��Python���(sh��)��(j��)�����wϵ 38 2.4.1 ��(g��u)���ij��� 38 2.4.2 ��(g��u)��˼· 39 2.4.3 �_�l(f��)���� 39 2.5 Python�W(xu��)��(x��)���Y(ji��) 44 2.5.1 �P(gu��n)��Python 45 2.5.2 Python����֪�R(sh��)�c(di��n) 45 ��3�� ��(sh��)��(j��)����ƪ 48 3.1 Hadoop�ηֲ�ʽ�İ��b���� 48 3.1.1 ����CentOS�h(hu��n)�� 48 3.1.2 ����Java�h(hu��n)�� 50 3.1.3 ����Hadoop�ηֲ�ʽ�h(hu��n)�� 51 3.2 ��(sh��)��(j��)�ھ��е�MapReduce���� 54 3.2.1 �W(xu��)��(x��)MapReduce���̵�Ŀ�� 54 3.2.2 MapReduce�Ĵ��aҎ(gu��)�� 55 3.2.3 ���εİ��� 58 3.3 ����MapReduce�еľ����� 60 3.3.1 ��ꇵĸ��� 60 3.3.2 ��ͬ��(ch��ng)���µľ����� 61 3.4 ��(sh��)��(j��)�ھ��е�Hive���� 67 3.4.1 ��ԇ�ĵ� 67 3.4.2 ��Python��(zh��)��HQL���� 67 3.4.3 ��֪��HQL֪�R(sh��) 69 3.5 ��(sh��)��(j��)�ھ��е�HBase���� 75 3.5.1 ֪�����P(gu��n)��ه�� 75 3.5.2 ��HBase�Ы@ȡ��(sh��)��(j��) 76 3.5.3 ��HBase�д惦(ch��)��(sh��)��(j��) 77 ��4�� ��(sh��)��(j��)�ھ���A(ch��)ƪ 81 4.1 MapReduce��Spark����(sh��)��(j��)�ھ�IJ 81 4.1.1 ���R(sh��)Hadoop���B(t��i)ϵ�y(t��ng) 81 4.1.2 ֪��Spark�����c(di��n) 83 4.1.3 ���̵IJ�� 85 4.1.4 ����֮�g���`���D(zhu��n)�Q 88 4.1.5 �x����m�Ĺ��� 89 4.2 ���(sh��)��(j��)�ھ��_�l(f��)�h(hu��n)�� 90 4.3 ��(d��ng)�֌�(sh��)�F(xi��n)�㷨���� 99 4.3.1 ֪��Spark On Yarn���\(y��n)��ģʽ 101 4.3.2 ��(chu��ng)��**��(g��)��(sh��)��(j��)�ھ��㷨 102 4.3.3 �������“����”���� 103 4.3.4 ��΄�(d��ng)�֌�(sh��)�F(xi��n)����ؐ�~˹�㷨 103 ��5�� ��(sh��)��(j��)�ھ��J(r��n)֪ƪ 107 5.1 ��Փ�c��(sh��)�`�IJ 107 5.2 ��(sh��)��(j��)�ھ��еĔ�(sh��)��(j��)��ϴ 110 5.2.1 ��(sh��)��(j��)��ϴ����Щ�� 110 5.2.2 ��(sh��)��(j��)�ıؚ��� 111 5.2.3 ��(sh��)�`�еĔ�(sh��)��(j��)��ϴ 112 5.3 ��(sh��)��(j��)�ھ��еĹ��߰� 120 5.3.1 �I(y��)��(w��)ģ���Ǻ��� 120 5.3.2 ����һ��(g��)�õ�ģ�� 121 ��6�� ��(sh��)��(j��)�ھ��㷨ƪ 123 6.1 �r(sh��)�g˥׃�㷨 123 6.1.1 �Ξ�r(sh��)�g˥׃ 123 6.1.2 ��������dȤ��ƫ�� 124 6.1.3 �r(sh��)�g˥׃�㷨�ij��� 124 6.1.4 ����Spark��(sh��)�F(xi��n)ģ�� 126 6.2 ��ֵ�� 130 6.2.1 �Ξ���Ϣ�� 130 6.2.2 ��ֵ���Č�(sh��)�F(xi��n)�^�� 130 6.2.3 �I(y��)��(w��)��(ch��ng)���Ľ�B 132 6.2.4 �㷨߉݋�ij��� 133 6.3 �A(y��)�y(c��)푑�(y��ng)�㷨 136 6.3.1 �I(y��)��(w��)��(ch��ng)���Ľ�B 136 6.3.2 ��(g��u)��ģ�͵�ǰ�ڹ��� 137 6.3.3 ���õ��A(y��)�y(c��)ģ�� 138 6.4 �Ӵη����㷨 140 6.5 �������������B(y��ng)�c��(sh��)�` 142 6.5.1 ������������Ҫ�� 142 6.5.2 ����Python��(sh��)�F(xi��n)�Ӵη����� 144 ��7�� �Ñ�����(sh��)�` 148 7.1 �Ñ�����đ�(y��ng)�È�(ch��ng)�� 148 7.1.1 �������� 148 7.1.2 �����{(di��o)�� 149 7.2 �Ñ�����Ę�(bi��o)���wϵ 150 7.2.1 ������� 151 7.2.2 ��(bi��o)���Ę�(g��u)�� 151 7.3 �Ñ������ģ�K��˼�S 152 7.3.1 �Ξ�ģ�K��˼�S 152 7.3.2 �Ñ������cģ�K��˼�S 153 7.4 �Ñ�����Ĺ����_�l(f��) 154 7.4.1 ��(du��)���_�l(f��)��ܵ��x�� 154 7.4.2 ģ�K�����ܵ��O(sh��)Ӌ(j��) 156 7.5 �Ñ���������ܠI�N 158 7.5.1 �I(y��)��(w��)�I�N 158 7.5.2 �I�N��(g��u)˼ 159 7.5.3 ���g(sh��)�y�c(di��n) 160 ��8�� �����p��(sh��)�`ƪ 162 8.1 “��ë�h”�O(ji��n)�صĘI(y��)��(w��) 162 8.1.1 “��ë�h”�Ķ��x�c���c(di��n) 162 8.1.2 “��ë”���ڵı�Ȼ�� 163 8.1.3 “��ë�h”���M(j��n)�� 164 8.1.4 “��ë�h”���ڵ����c�� 165 8.1.5 “��ë�h”�O(ji��n)��ƽ�_(t��i)�����x 165 8.2 “��ë�h”�O(ji��n)�ص��O(sh��)��ָ�y 166 8.2.1 �Ξ��O(sh��)��ָ�y 166 8.2.2 �׌Ӆ���(sh��) 167 8.2.3 ��(y��ng)�È�(ch��ng)�� 168 8.2.4 �Ƅ�(d��ng)�˵Ĕ�(sh��)��(j��)�־û� 169 8.2.5 �O(sh��)��ָ�y�����㷨 169 8.3 “��ë�h”�O(ji��n)�صĔ�(sh��)��(j��)�(q��)��(d��ng) 170 8.3.1 �O(ji��n)�ص�Ŀ�� 170 8.3.2 ��(sh��)��(j��)���“ʳ��” 172 8.4 “��ë�h”�O(ji��n)�صČ�(sh��)�`���� 173 ��9�� ��(sh��)��(j��)�ھ��`��ƪ 178 9.1 ��Ώ�0��1�D(zhu��n)�͵���(sh��)��(j��)Ȧ�� 178 9.2 ��(sh��)��(j��)�ھ�ĘI(y��)�߾C�������u(p��ng)�� 180 9.2.1 �����ij��� 180 9.2.2 �C�������u(p��ng)�� 181 9.2.3 ��(g��)��ָ��(bi��o)�wϵ����(sh��)��(j��)�ھ� 182 9.3 �o��Ҫ�M(j��n)�딵(sh��)��(j��)�ھ�Ȧ�ӵ�����һ�c(di��n)���h 183 9.3.1 �\���c���b 184 9.3.2 �I������ 185 9.3.3 Ͷ�ú��v 186 9.3.4 ������ԇ 186 9.3.5 �Y(ji��)β 187 ��ӛ ��(sh��)��(j��)�r(ji��)ֵ̽���c��(sh��)��(j��)�a(ch��n)Ʒ��(sh��)�` 188