Question

We will be using the Map-Reduce framework using Hadoop streaming. You are expected to implement Mapper and Runner components in any language, with the runner script that calls these in python Given two matrices A of size m ∗ n and B of size n ∗ p. Output the matrix multiplication of A and B.

Answer 1

To perform matrix multiplication using Hadoop streaming and the Map-Reduce framework, you need to implement the Mapper and Reducer components. The Mapper emits intermediate key-value pairs, while the Reducer calculates the final matrix multiplication result. The runner script in Python can be used to call these components.

Step-by-step explanation:

In order to perform matrix multiplication using the Map-Reduce framework and Hadoop streaming, you will need to implement the Mapper and Reducer components. The Mapper takes a pair of input matrices and emits intermediate key-value pairs, while the Reducer receives these pairs and computes the final matrix multiplication result. The runner script in Python can be used to call these components and manage the overall execution.

To implement the Mapper, you can use any programming language of your choice. The Mapper will receive input records containing elements from Matrix A and Matrix B, and it needs to emit intermediate key-value pairs such that the keys represent the positions of elements in the resulting matrix, and the values contain the multiplying factors.

The Reducer will receive the intermediate key-value pairs and perform the actual matrix multiplication calculation. It will receive all values for a specific key, which represent the multiplying factors for a specific position in the resulting matrix. The Reducer then sums all these values and emits the final result.