Let \(\vec{a}=\overset{^}{i}+2\overset{^}{j}+3\overset{^}{k},\vec{b}=3\overset{^}{i}+\overset{^}{j}-\overset{^}{k}\) and…

\(\text{Given, }\vec{a}=\overset{^}{i}+2\overset{^}{j}+3\overset{^}{k}&\vec{b}=3\overset{^}{i}+\overset{^}{j}-\overset{^}{k}\\ \vec{a}\cdot \vec{b}=2&\vec{a}\cdot \vec{c}=5&\vec{b}\cdot \vec{c}=0\\ |\vec{a}|=\sqrt{14}\text{ and }|\vec{b}|=\sqrt{11}\\ \vec{c}=\lambda \vec{a}+\mu \vec{b}\\ \vec{a}\cdot \vec{c}=\lambda |\vec{a}{|}^{2}+\mu \vec{a}\cdot \vec{b}\\ 5=\lambda \times 14+\mu \times 2........\text{(i)}\\ \vec{b}\cdot \vec{c}=\lambda \vec{a}\cdot \vec{b}+\mu (\vec{b}{)}^{2}\\ 0=2\lambda +11\mu ........\text{.(ii)}\\ \text{On solving, (i) & (ii), we get}\\ \mu =-\frac{1}{15}\lambda =\frac{11}{30}\\ \text{Now},\vec{c}=\frac{1}{6}\overset{^}{i}+\frac{2}{3}\overset{^}{j}+\frac{7}{6}\overset{^}{k}\\ |\vec{c}|=\sqrt{\frac{11}{6}}\)