Answer:
coefficient of xy^2 is : 864 and that of y^3 is 1728
Explanation:
Using binomial theorem,
1st term is 1 (ax)^3 = 8x^3 from which a = 2
2nd term is 3(ax)^2(by) = -144x^2y from which b = -12
3rd term is 3(ax)(by)^2 = 3(2x)(-12y)^2 = 864
4th term is 1(by)^3 = 1(-12y)^3 = -1728y^3