5
5
6
6
Mars is a tensor-based unified framework for large-scale data computation.
7
7
8
+ Installation
9
+ ------------
10
+
11
+ Mars is easy to install by
12
+
13
+ .. code-block :: bash
14
+
15
+ pip install pymars
16
+
17
+ The distributed version can be installed by
18
+
19
+ .. code-block :: bash
20
+
21
+ pip install ' pymars[distributed]'
22
+
23
+ For now, distributed version is only available on Linux and Mac OS.
24
+
25
+
8
26
Mars tensor
9
27
-----------
10
28
@@ -56,6 +74,102 @@ Mars can scale in to a single machine, and scale out to a cluster with thousands
56
74
Both the local and distributed version share the same piece of code,
57
75
it's fairly simple to migrate from a single machine to a cluster due to the increase of data.
58
76
77
+ Running on a single machine including thread-based scheduling,
78
+ local cluster scheduling which bundles the whole distributed components.
79
+ Mars is also easy to scale out to a cluster by starting different components of
80
+ mars distributed runtime on different machines in the cluster.
81
+
82
+ Threaded
83
+ ````````
84
+
85
+ ``execute `` method will by default run on the thread-based scheduler on a single machine.
86
+
87
+ .. code-block :: python
88
+
89
+ import mars.tensor as mt
90
+
91
+ a = mt.ones(10 , 10 )
92
+ a.execute()
93
+
94
+ Users can create a session explicitly.
95
+
96
+ .. code-block :: python
97
+
98
+ from mars.session import new_session
99
+
100
+ session = new_session()
101
+ session.run(a + 1 )
102
+ (a * 2 ).execute(session = session)
103
+
104
+ # session will be released when out of with statement
105
+ with new_session() as session2:
106
+ session2.run(a / 3 )
107
+
108
+
109
+ Local cluster
110
+ `````````````
111
+
112
+ Users can start the local cluster bundled with the distributed runtime on a single machine.
113
+ Local cluster mode requires mars distributed version.
114
+
115
+ .. code-block :: python
116
+
117
+ from mars.deploy.local import new_cluster
118
+
119
+ # cluster will create a session and set it as default
120
+ cluster = new_cluster()
121
+
122
+ # run on the local cluster
123
+ (a + 1 ).execute()
124
+
125
+ # create a session explicitly by specifying the cluster's endpoint
126
+ session = new_session(cluster.endpoint)
127
+ session.run(a * 3 )
128
+
129
+
130
+ Distributed
131
+ ```````````
132
+
133
+ After installing the distributed version on every node in the cluster,
134
+ A node can be selected as scheduler and another as web service,
135
+ leaving other nodes as workers. The scheduler can be started with the following command:
136
+
137
+ .. code-block :: bash
138
+
139
+ mars-scheduler -a < scheduler_ip> -p < scheduler_port>
140
+
141
+ Web service can be started with the following command:
142
+
143
+ .. code-block :: bash
144
+
145
+ mars-web -a < web_ip> -s < scheduler_ip> --ui-port < ui_port_exposed_to_user>
146
+
147
+ Workers can be started with the following command:
148
+
149
+ .. code-block :: bash
150
+
151
+ mars-worker -a < worker_ip> -p < worker_port> -s < scheduler_ip>
152
+
153
+ After all mars processes are started, users can run
154
+
155
+ .. code-block :: python
156
+
157
+ sess = new_session(' http://<web_ip>:<ui_port>' )
158
+ a = mt.ones((2000 , 2000 ), chunks = 200 )
159
+ b = mt.inner(a, a)
160
+ sess.run(b)
161
+
162
+
163
+ Getting involved
164
+ ----------------
165
+
166
+ - Join the mailing list: send an email to `mars-dev@googlegroups.com `_.
167
+ - Please report bugs by submitting a `GitHub issue `_.
168
+ - Submit contributions using `pull requests `_.
169
+
170
+ Thank you in advance for your contributions!
171
+
172
+
59
173
.. |Build | image :: https://img.shields.io/travis/mars-project/mars.svg?style=flat-square
60
174
:target: https://travis-ci.org/mars-project/mars
61
175
.. |Coverage | image :: https://img.shields.io/coveralls/github/mars-project/mars.svg?style=flat-square
@@ -67,3 +181,6 @@ it's fairly simple to migrate from a single machine to a cluster due to the incr
67
181
.. |License | image :: https://img.shields.io/pypi/l/pymars.svg?style=flat-square
68
182
:target: https://github.com/mars-project/mars/blob/master/LICENSE
69
183
.. |Implementation | image :: https://img.shields.io/pypi/implementation/pymars.svg?style=flat-square
184
+ .. _`mars-dev@googlegroups.com` : https://groups.google.com/forum/#!forum/mars-dev
185
+ .. _`GitHub issue` : https://github.com/mars-project/mars/issues
186
+ .. _`pull requests` : https://github.com/mars-project/mars/pulls
0 commit comments