To calculate the cosine of a determined angle of a rigth triangle without knowing the measure of the angle you have to make use of the trigonometrical ration.
The cosine of a certain angle θ is equal to the ratio between the side sdjacent to said angle and the hypothenuse of the triangle.
For the given triangle the side adjacent to ∠A is AC=5 and the hypothenuce is AB=13
THe cosine of A is then: