Big Data Programming Map Reduce Please answer this question. Please provide the description of your design, your source code, a screen shot showing the command you use to start your job, a screenshot showing results.
Here is the question (photos are attached and the matrix/code is also attached):
Recall in BDP-3-mapReduce-3 slides, we discussed the situation where vector v is
too big to fit into the memory (slide 6). To address this situation, we will need to
divide matrix M into vertical stripes, and also divide v into equal number of
horizontal stripes:
Your task is to design a mapReduce algorithm and implement the algorithm to
divide matrix M into vertical stripes.
Again, matrix M is stored in a file, using (i, j,mij) format. Here is one example matrix,
1 7
11 12 16
22 23 28
30 34
42 45
51 56
63 67
71 78
80 82 84 86 88
and the actual file that stores this matrix is given by the following (you can find this
file from our course website).
(0,1,1)
(8,0,80)
(0,7,7)
(1,1,11)
(3,0,30)
(2,2,22)
(8,4,84)
(2,8,28)
(3,4,34)
(6,3,63)
(4,5,45)
(5,1,51)
(8,8,88)
(5,6,56)
(6,7,67)
(7,1,71)
(1,6,16)
(7,8,78)
(1,2,12)
(8,2,82)
(2,3,23)
(8,6,86)
(4,2,42)
As you can tell, this matrix is 9 x 9, and we would like to break it into 3 vertical
stripes (therefore each stripe is 9 x 3). The final result will be 3 separate files, each
file holds one stripe from the original matrix.
You will need to submit the following:
a very brief description of your design. The description should be in this
format
map task: for each …, emits …,
reduce task: …
your source code (30 points)
a screenshot showing the command you use to start your job (5 points)
a screenshot showing the result of your submitted job, should be something
look like this (5 points)
hint: you will need to write a customer partitioner, something like this:
public static class extends
Partitioner