Find b and c so that (5, b, c. is orthogonal to both (1, 5, 3) and (1, −5, 1).

A vector orthogonal to two others (in 3-d space) can be obtained by the cross product.

A=
x y z
1 5 3
1 -5 1

Evaluate A
(5+15,-1*(1)+1*3,1(-5)-1(5))
=(20,2,10)
=(5,1/2,-5/2)

Check: 
(5,1/2,-5/2).(1,5,3)=0 
(5,1/2,-5/2).(1,-5,1)=0
So solution is good.

b=1/2, c=-5/2

Alternatively, by orthogonal conditions,
multiply (5,b,c) by (1,5,3) = (3c+5b+5) => 3c+5b+5=0(1)
multiply (5,b,c) by (1,-5,1) = (c-5b+5) => c-5b+5=0(2)
Solve (1) & (2) for b and c
(1)+(2) => 4c+10=0 => c=-5/2
Substitute c in 2
-5/2-5b+5=0 => b=1/2
So we get the same results.