The domain of the function in interval notation is (-1, 5], which is the second option
The steps used to find the domain of the function are as follows;
The domain is the set of the possible x-values of a function. The function in the question is shown in the graph
The x-values are the values of the function describing the relative horizontal spatial location of the values describing the function
The range can be described by the values with the horizontal stretch of the function. The values in the left and rightmost reach of the function in the graph are; (-1, 5), and (5, 5)
The value in the left end is marked by an open circle, indicating that the function is not defined at (-1, 5), or the point (-1, 5) is excluded from the values of the function such that the domain is represented in interval notation using a left parenthesis on the left
The value on the rightmost part is (5, 5), marked with a closed or shaded circle, indicating that (5, 5) is a value of the function, therefore, the domain is represented by a square bracket on the right, to get the domain in interval notation ad (-1, 5]