JEE Mathematics Trigonometric Functions MCQs Set 20

Answer: (b) \( \cos a,\cos b,\cos c \)

Explanation:
1. Divide the first equation by \( \sin a\ne0 \), using \( \sin2a=2\sin a\cos a \), \( \sin3a=\sin a(4\cos^2a-1) \) and \( \sin4a=\sin a\,(8\cos^3a-4\cos a) \).
2. We get \( x+2y\cos a+z(4\cos^2a-1)=8\cos^3a-4\cos a \), that is \( 8\cos^3a-4z\cos^2a-(2y+4)\cos a+(z-x)=0 \).
3. Dividing by 8: \( \cos^3a-\frac z2\cos^2a-\frac{y+2}4\cos a+\frac{z-x}8=0 \). So \( \cos a \) satisfies the given cubic, and by the same argument so do \( \cos b \) and \( \cos c \).

Teacher's Note:
1. Reduce each equation by dividing out \( \sin\theta \) and writing every term as a polynomial in \( \cos\theta \).
2. \( \sin3a=3\sin a-4\sin^3a=\sin a(4\cos^2a-1) \).

Answer: (a) \( (a-b)^2 \)

Explanation:
1. Squaring: \( u^2=a^2+b^2+2\sqrt{(a^4+b^4)\cos^2\theta\sin^2\theta+a^2b^2(\cos^4\theta+\sin^4\theta)} \).
2. The expression under the root simplifies to \( (a^2-b^2)^2\frac{\sin^22\theta}4+a^2b^2 \). As \( 0\le\sin^22\theta\le1 \), it lies between \( a^2b^2 \) and \( \frac{(a^2-b^2)^2}4+a^2b^2=\frac{(a^2+b^2)^2}4 \).
3. So \( u^2_{\min}=a^2+b^2+2|ab|=(|a|+|b|)^2 \) and \( u^2_{\max}=a^2+b^2+(a^2+b^2)=2(a^2+b^2) \). The difference is \( 2(a^2+b^2)-(a+b)^2=(a-b)^2 \) (for \( a,b>0 \)).

Teacher's Note:
1. Use \( \cos^4\theta+\sin^4\theta=1-\frac12\sin^22\theta \) to express everything with \( \sin^22\theta \).
2. Test: \( a=2,b=1 \) gives maximum 10 and minimum 9, difference \( 1=(2-1)^2 \).

Answer: (c) \( \frac\pi{12} \)

Explanation:
1. Use \( \operatorname{cosec}A\operatorname{cosec}B=\frac{\cot A-\cot B}{\sin(B-A)} \) with \( B-A=\frac\pi4 \) and \( \sin\frac\pi4=\frac1{\sqrt2} \): each term equals \( \sqrt2\left[\cot\left(\theta+\frac{(m-1)\pi}4\right)-\cot\left(\theta+\frac{m\pi}4\right)\right] \).
2. The sum telescopes: \( \sqrt2\left[\cot\theta-\cot\left(\theta+\frac{3\pi}2\right)\right]=\sqrt2(\cot\theta+\tan\theta) \). Setting it equal to \( 4\sqrt2 \): \( \cot\theta+\tan\theta=4 \).
3. So \( \frac2{\sin2\theta}=4 \), \( \sin2\theta=\frac12 \), \( 2\theta=\frac\pi6 \) or \( \frac{5\pi}6 \), giving \( \theta=\frac\pi{12} \) or \( \frac{5\pi}{12} \). Among the options, \( \frac\pi{12} \) is a solution (\( \frac{7\pi}{12} \) is outside the interval).

Teacher's Note:
1. Write \( \frac1{\sin A\sin B} \) as a difference of cotangents to telescope the sum.
2. \( \cot\left(\theta+\frac{3\pi}2\right)=-\tan\theta \).

Answer: (c) 0

Explanation:
1. Multiply by \( \sin x\cos x \): \( \sqrt3\sin x+\cos x+2(\sin^2x-\cos^2x)=0 \), i.e. \( \sqrt3\sin x+\cos x=2\cos2x \).
2. Dividing by 2: \( \cos\left(x-\frac\pi3\right)=\cos2x \), so \( x-\frac\pi3=2n\pi\pm2x \), giving \( x=-2n\pi-\frac\pi3 \) or \( x=\frac{2n\pi}3+\frac\pi9 \).
3. In \( (-\pi,\pi) \): \( x=-\frac\pi3 \), and \( x=\frac\pi9,\ \frac{7\pi}9,\ -\frac{5\pi}9 \). None of these is \( 0,\pm\frac\pi2 \).
4. Sum \( =-\frac\pi3+\frac\pi9+\frac{7\pi}9-\frac{5\pi}9=0 \).

Teacher's Note:
1. Clear denominators and use the auxiliary angle form \( a\sin x+b\cos x \).
2. Reject any solution at which the original expression is undefined.

Answer: (a) \( \frac1{2^{n/2}} \)

Explanation:
1. The condition gives \( \prod\cos\alpha_i=\prod\sin\alpha_i \). Let \( y=\prod\cos\alpha_i \). Then \( y^2=\prod\cos\alpha_i\sin\alpha_i=\frac1{2^n}\prod\sin2\alpha_i \).
2. Since \( 0\le2\alpha_i\le\pi \), \( 0\le\sin2\alpha_i\le1 \), so \( y^2\le\frac1{2^n} \).
3. Hence \( y\le\frac1{2^{n/2}} \), with equality when every \( \alpha_i=\frac\pi4 \).

Teacher's Note:
1. Use the cot condition to replace one copy of the cosine product by the sine product.
2. Equality holds when all angles are \( \frac\pi4 \) (each \( \cos\frac\pi4=\frac1{\sqrt2} \)).

Answer: (b) \( 2\sqrt{1+k} \)

Explanation:
1. The angles with sine \( k \) in ascending order are \( \alpha \), \( \beta=\pi-\alpha \), \( \gamma=2\pi+\alpha \), \( \delta=3\pi-\alpha \).
2. Then \( \sin\frac\beta2=\cos\frac\alpha2 \), \( \sin\frac\gamma2=\sin\left(\pi+\frac\alpha2\right)=-\sin\frac\alpha2 \), \( \sin\frac\delta2=\sin\left(\frac{3\pi}2-\frac\alpha2\right)=-\cos\frac\alpha2 \).
3. The expression becomes \( 4\sin\frac\alpha2+3\cos\frac\alpha2-2\sin\frac\alpha2-\cos\frac\alpha2=2\left(\sin\frac\alpha2+\cos\frac\alpha2\right) \).
4. \( =2\sqrt{1+2\sin\frac\alpha2\cos\frac\alpha2}=2\sqrt{1+\sin\alpha}=2\sqrt{1+k} \).

Teacher's Note:
1. List the principal solutions of \( \sin\theta=k \) in order: \( \alpha,\pi-\alpha,2\pi+\alpha,3\pi-\alpha \).
2. Combine \( \sin\frac\alpha2+\cos\frac\alpha2 \) by squaring.

Answer: (d) All of (i), (ii), (iii) and (iv)

Explanation:
1. \( 1+\sec\theta=\frac{1+\cos\theta}{\cos\theta}=\frac{2\cos^2\frac\theta2}{\cos\theta} \), and similarly \( 1+\sec2^k\theta=\frac{2\cos^22^{k-1}\theta}{\cos2^k\theta} \).
2. The product telescopes: \( f_n(\theta)=\tan\frac\theta2\cdot\frac{2\cos^2\frac\theta2}{\cos\theta}\cdot\frac{2\cos^2\theta}{\cos2\theta}\cdots\frac{2\cos^22^{n-1}\theta}{\cos2^n\theta}=\tan2^n\theta \).
3. So \( f_2\left(\frac\pi{16}\right)=\tan\frac{4\pi}{16}=1 \), \( f_3\left(\frac\pi{32}\right)=\tan\frac{8\pi}{32}=1 \), \( f_4\left(\frac\pi{64}\right)=\tan\frac{16\pi}{64}=1 \), \( f_5\left(\frac\pi{128}\right)=\tan\frac{32\pi}{128}=1 \). All four are true.

Teacher's Note:
1. Using \( \sin2x=2\sin x\cos x \) repeatedly makes the product telescope into \( \tan2^n\theta \).
2. Check: with \( n=0 \), \( \tan\frac\theta2(1+\sec\theta)=\tan\theta \).

Answer: (a) 13 if \( \tan\beta>0 \), and \( \frac{119+120\cot\alpha}{13} \) if \( \tan\beta<0 \)

Explanation:
1. Expand: \( 5\sin(\alpha+\beta)-12\cos(\alpha+\beta)=(5\cos\beta+12\sin\beta)\sin\alpha+(5\sin\beta-12\cos\beta)\cos\alpha \). Here \( \sin\beta=\frac{12}{13} \) and \( \cos\beta=\pm\frac5{13} \).
2. If \( \tan\beta>0 \), then \( \cos\beta=\frac5{13} \): the expression is \( \left(\frac{25+144}{13}\right)\sin\alpha+\left(\frac{60-60}{13}\right)\cos\alpha=13\sin\alpha \), so multiplying by \( \operatorname{cosec}\alpha \) gives 13.
3. If \( \tan\beta<0 \), then \( \cos\beta=-\frac5{13} \): the expression is \( \frac{119}{13}\sin\alpha+\frac{120}{13}\cos\alpha \), and multiplying by \( \operatorname{cosec}\alpha \) gives \( \frac{119+120\cot\alpha}{13} \).

Teacher's Note:
1. Because \( 0<\beta<\pi \) with \( \sin\beta>0 \), the sign of \( \tan\beta \) tells you the sign of \( \cos\beta \).
2. Do not forget the factor \( \operatorname{cosec}\alpha \) at the end.

Answer: (a) (ii), (iii) and (iv) only

Explanation:
1. Expand the first relation: \( a[\sin(\theta+\phi)-\sin(\theta-\phi)]=b[\sin(\theta+\phi)+\sin(\theta-\phi)] \), i.e. \( 2a\cos\theta\sin\phi=2b\sin\theta\cos\phi \), so \( a\tan\phi=b\tan\theta \). Statement (ii) is true and (i) is false.
2. In terms of half-angles, \( \frac{2a\tan\frac\phi2}{1-\tan^2\frac\phi2}=\frac{2b\tan\frac\theta2}{1-\tan^2\frac\theta2} \). Substituting \( \tan\frac\theta2=\frac1a\left(b\tan\frac\phi2+c\right) \) and simplifying gives \( \tan\frac\phi2(a^2-b^2-c^2)=bc\left(1+\tan^2\frac\phi2\right) \).
3. Hence \( \sin\phi=\frac{2\tan\frac\phi2}{1+\tan^2\frac\phi2}=\frac{2bc}{a^2-b^2-c^2} \) (iii). By symmetry, \( \sin\theta=\frac{2ac}{a^2-b^2+c^2} \) (iv). A numerical check with \( a=3,\ b=1.2,\ \theta=0.5 \) agrees with all three results.

Teacher's Note:
1. Write the first relation as a ratio of sine of sum to sine of difference, then use componendo-dividendo.
2. Verify with numbers when a long algebraic result is involved.

Answer: (a) (ii) and (iii) only

Explanation:
1. \( \frac{\tan3A-\tan A}{\tan A}=k-1\Rightarrow\frac{\sin2A}{\cos3A\sin A}=k-1\Rightarrow\frac{2\cos A}{\cos3A}=k-1 \), so \( \frac{\cos A}{\cos3A}=\frac{k-1}2 \). Statement (i) is false.
2. \( \frac{\sin3A}{\sin A}=k\cdot\frac{\cos3A}{\cos A}=k\cdot\frac2{k-1}=\frac{2k}{k-1} \). Statement (ii) is true.
3. Also \( \frac{\sin3A}{\sin A}=3-4\sin^2A \), so \( 4\sin^2A=\frac{k-3}{k-1} \). From \( 0<\sin^2A<1 \): \( 0<\frac{k-3}{k-1}<4 \). The first inequality gives \( k<1 \) or \( k>3 \); the second gives \( \frac{3k-1}{k-1}>0 \), i.e. \( k<\frac13 \) or \( k>1 \). Together, \( k<\frac13 \) or \( k>3 \). Statement (iii) is true and (iv) is false.

Teacher's Note:
1. Use \( \sin3A=3\sin A-4\sin^3A \) and the condition \( 0<\sin^2A<1 \) to find the permissible values of \( k \).
2. Check with \( A=1 \): \( k=\frac{\tan3}{\tan1}\approx-0.092<\frac13 \), consistent with (iii).

Answer: (a) 4

Explanation:
1. By the Cauchy-Schwarz inequality, \( (a\tan\alpha+\sqrt{a^2-1}\tan\beta+\sqrt{a^2+1}\tan\gamma)^2\le\left[a^2+(a^2-1)+(a^2+1)\right]\left(\tan^2\alpha+\tan^2\beta+\tan^2\gamma\right) \).
2. That is \( 4a^2\le3a^2\left(\tan^2\alpha+\tan^2\beta+\tan^2\gamma\right) \), so \( \tan^2\alpha+\tan^2\beta+\tan^2\gamma\ge\frac43 \).
3. Hence \( 3\left(\tan^2\alpha+\tan^2\beta+\tan^2\gamma\right)\ge4 \); the least value is 4.

Teacher's Note:
1. Cauchy-Schwarz bounds a linear combination by the product of two sums of squares.
2. The sum of the squares of the coefficients is \( 3a^2 \).

Answer: (a) 3

Explanation:
1. \( \tan^2\frac\alpha2=\frac{1-\cos\alpha}{1+\cos\alpha}=\frac{(2-\cos\beta)-(2\cos\beta-1)}{(2-\cos\beta)+(2\cos\beta-1)}=\frac{3-3\cos\beta}{1+\cos\beta}=3\tan^2\frac\beta2 \).
2. So \( \frac{\tan\frac\alpha2}{\tan\frac\beta2}=\sqrt3 \) (both half-angle tangents are positive as \( 0<\alpha<\beta<\pi \)).
3. Therefore \( \sqrt3\times\sqrt3=3 \).

Teacher's Note:
1. Use \( \tan^2\frac\alpha2=\frac{1-\cos\alpha}{1+\cos\alpha} \) directly.
2. Both half-angles lie in \( \left(0,\frac\pi2\right) \), so the tangents are positive.

Answer: (a) 0

Explanation:
1. Let the common value be \( k \). Then \( \frac1x=\frac{\cos\theta}k,\ \frac1y=\frac{\cos(\theta+\frac{2\pi}3)}k,\ \frac1z=\frac{\cos(\theta+\frac{4\pi}3)}k \).
2. \( \frac1x+\frac1y+\frac1z=\frac1k\left[\cos\theta+\cos\left(\theta+\frac{2\pi}3\right)+\cos\left(\theta+\frac{4\pi}3\right)\right]=0 \), since the three cosines sum to zero.
3. \( xy+yz+zx=xyz\left(\frac1x+\frac1y+\frac1z\right)=0 \).

Teacher's Note:
1. Express \( \frac1x,\frac1y,\frac1z \) in terms of the common value and use \( \sum\cos\left(\theta+\frac{2k\pi}3\right)=0 \).
2. \( xy+yz+zx=xyz\sum\frac1x \).

Answer: (a) 6

Explanation:
1. \( \sin^3x=\frac{3\sin x-\sin3x}4 \), so \( \sin^3x\sin3x=\frac34\sin x\sin3x-\frac14\sin^23x \).
2. \( \sin x\sin3x=\frac12(\cos2x-\cos4x) \) and \( \sin^23x=\frac12(1-\cos6x) \).
3. So \( \sin^3x\sin3x=\frac38(\cos2x-\cos4x)-\frac18(1-\cos6x)=-\frac18+\frac38\cos2x-\frac38\cos4x+\frac18\cos6x \). The highest multiple is 6 and its coefficient is non-zero, so \( n=6 \).

Teacher's Note:
1. Reduce powers and products to cosines of multiple angles.
2. The highest frequency present gives \( n \).

Answer: (b) \( 3(m^2-1)^2 \)

Explanation:
1. \( P_1^2=m^2\Rightarrow1+2\sin\theta\cos\theta=m^2 \), so \( \sin\theta\cos\theta=\frac{m^2-1}2 \).
2. \( P_6=\sin^6\theta+\cos^6\theta=1-3\sin^2\theta\cos^2\theta \), so \( 1-P_6=3\left(\sin\theta\cos\theta\right)^2=\frac{3(m^2-1)^2}4 \).
3. Hence \( 4(1-P_6)=3(m^2-1)^2 \).

Teacher's Note:
1. \( a^3+b^3=(a+b)^3-3ab(a+b) \) with \( a=\sin^2\theta,\ b=\cos^2\theta \) gives \( P_6=1-3\sin^2\theta\cos^2\theta \).
2. Find \( \sin\theta\cos\theta \) from \( P_1^2 \).

Answer: (a) 8

Explanation:
1. Let \( k=\sin^2\theta\cos^2\theta \). The recurrence is \( P_n=P_{n-2}-kP_{n-4} \) (since \( \sin^n\theta+\cos^n\theta=(\sin^{n-2}\theta+\cos^{n-2}\theta)-\sin^2\theta\cos^2\theta(\sin^{n-4}\theta+\cos^{n-4}\theta) \)).
2. \( P_2=1,\ P_4=1-2k,\ P_6=1-3k \). Then \( P_8=P_6-kP_4=1-3k-k(1-2k)=1-4k+2k^2 \) and \( P_{10}=P_8-kP_6=1-4k+2k^2-k(1-3k)=1-5k+5k^2 \).
3. \( 6P_{10}-15P_8+10P_6+7=6(1-5k+5k^2)-15(1-4k+2k^2)+10(1-3k)+7=(6-15+10+7)+(-30+60-30)k+(30-30)k^2=8 \).

Teacher's Note:
1. Find a recurrence for \( P_n \) in terms of \( k=\sin^2\theta\cos^2\theta \).
2. All \( k \) and \( k^2 \) terms cancel, so the value is a constant.

Answer: (b) \( 1-\frac{\sqrt{69}}6 \)

Explanation:
1. Equation (1): write \( 2=2(\sin^2x+\cos^2x) \): \( 3\sin^2x+3\sin x\cos x-5\cos^2x=0 \), i.e. \( 3\tan^2x+3\tan x-5=0 \), so \( \tan\alpha=\frac{-3\pm\sqrt{69}}6 \).
2. Equation (2): using \( \cos2x=\cos^2x-\sin^2x,\ \sin2x=2\sin x\cos x \) and \( 2=2(\sin^2x+\cos^2x) \) it becomes \( 3\sin^2x+2\sin x\cos x-3\cos^2x=... \) which, after simplification, is \( \cos x(2\sin x-3\cos x)=0 \). So \( \cos x=0 \) (tangent undefined) or \( \tan x=\frac32 \); hence \( \tan\beta=\frac32 \).
3. \( \tan\alpha+\tan\beta=\frac{-3\pm\sqrt{69}}6+\frac32=\frac{6\pm\sqrt{69}}6=1\pm\frac{\sqrt{69}}6 \). Among the options, \( 1-\frac{\sqrt{69}}6 \) occurs.

Teacher's Note:
1. Convert both equations to equations in \( \tan x \) by dividing by \( \cos^2x \) after making them homogeneous.
2. Check that the chosen value of \( \tan\beta \) satisfies equation (2): \( x=\tan^{-1}1.5 \) gives 0.

Answer: (d) \( -\frac53-\frac2{\sqrt{13}} \)

Explanation:
1. Equation (1) is \( 3\tan^2x+3\tan x-5=0 \), whose roots \( \tan\alpha,\tan\beta \) have product \( \tan\alpha\tan\beta=-\frac53 \).
2. Equation (2) reduces to \( \cos x(2\sin x-3\cos x)=0 \), so \( \cos x=0 \) or \( \tan x=\frac32 \), i.e. \( \cos x=\pm\frac2{\sqrt{13}} \). So \( \cos\gamma,\cos\delta\in\left\{0,\pm\frac2{\sqrt{13}}\right\} \).
3. Taking \( \cos\gamma=0 \) and \( \cos\delta=-\frac2{\sqrt{13}} \): the expression is \( -\frac53-\frac2{\sqrt{13}} \), which is option (d). The other options cannot be obtained.

Teacher's Note:
1. Use the product of roots \( \frac ca \) for the quadratic in \( \tan x \).
2. List all possible values of \( \cos x \) from equation (2) before choosing the sum.

Answer: (a) 9

Explanation:
1. Expand: \( f=\sin^2\theta+\operatorname{cosec}^2\theta+2+\cos^2\theta+\sec^2\theta+2=1+4+\operatorname{cosec}^2\theta+\sec^2\theta \).
2. \( \operatorname{cosec}^2\theta+\sec^2\theta=2+\cot^2\theta+\tan^2\theta \). So \( f=7+\cot^2\theta+\tan^2\theta=9+(\tan\theta-\cot\theta)^2 \).
3. The minimum value is 9, attained at \( \tan\theta=\cot\theta \), i.e. \( \theta=\frac\pi4 \).

Teacher's Note:
1. Write the squares of the reciprocal functions with \( 1+\tan^2 \) and \( 1+\cot^2 \).
2. \( \tan^2\theta+\cot^2\theta\ge2 \) by AM-GM.

Answer: (a) 4

Explanation:
1. \( a^2+b^2=\sin^2\alpha+\sin^2\beta-2\sin\alpha\sin\beta+\cos^2\alpha+\cos^2\beta+2\cos\alpha\cos\beta=2+2\cos(\alpha+\beta) \).
2. \( =4\cos^2\frac{\alpha+\beta}2\le4 \).
3. So the maximum value is 4.

Teacher's Note:
1. Pair the cross terms to form \( \cos(\alpha+\beta) \).
2. The maximum of \( \cos^2 \) is 1.

Answer: (a) 1

Explanation:
1. The sine function is concave on \( \left[0,\frac\pi2\right] \), so \( \frac{\sin A+\sin B}2\le\sin\frac{A+B}2=\sin\frac\pi4 \).
2. Hence \( \sin A+\sin B\le2\sin\frac\pi4=\sqrt2 \), with equality at \( A=B=\frac\pi4 \).
3. Therefore \( (\sin A+\sin B)\cos\frac\pi4\le\sqrt2\cdot\frac1{\sqrt2}=1 \); the maximum value is 1.

Teacher's Note:
1. Jensen's inequality for a concave function gives the bound.
2. Alternatively \( \sin A+\cos A=\sqrt2\sin\left(A+\frac\pi4\right)\le\sqrt2 \).

Answer: (a) 2 or 6

Explanation:
1. Write \( 7\cos x+6\sin x=6(2\cos x+\sin x)-5\cos x=6-5\cos x \).
2. From \( \sin x=1-2\cos x \): \( 1-\cos^2x=1-4\cos x+4\cos^2x \), so \( 5\cos^2x-4\cos x=0 \), giving \( \cos x=0 \) or \( \cos x=\frac45 \).
3. Both satisfy the original equation (\( \sin x=1 \) and \( \sin x=-\frac35 \)). The value is \( 6-0=6 \) or \( 6-4=2 \).

Teacher's Note:
1. Rewrite the target expression using the given equation to reduce it to one trigonometric function.
2. Squaring can add extra roots, so verify each solution (here both are valid).

Answer: (a) \( \left[\frac34,1\right] \)

Explanation:
1. \( y=\cos^2\theta+\sin^4\theta=\cos^2\theta+\sin^2\theta(1-\cos^2\theta)=1-\sin^2\theta\cos^2\theta \).
2. \( =1-\frac14\sin^22\theta \). Since \( 0\le\sin^22\theta\le1 \), \( \frac34\le y\le1 \).
3. Both ends are attained, so the range is \( \left[\frac34,1\right] \).

Teacher's Note:
1. Express the function through \( \sin^22\theta \).
2. Check: \( \theta=\frac\pi4 \) gives \( \frac12+\frac14=\frac34 \).

Answer: (a) \( (0,1) \)

Explanation:
1. \( \tan A<0 \) means \( A>\frac\pi2 \), so \( B+C<\frac\pi2 \) with both \( B,C \) acute (\( \tan B,\tan C>0 \)).
2. \( \tan(B+C)>0 \), so \( \frac{\tan B+\tan C}{1-\tan B\tan C}>0 \). The numerator is positive, hence \( 1-\tan B\tan C>0 \).
3. Therefore \( 0<\tan B\tan C<1 \).

Teacher's Note:
1. An obtuse angle forces the other two angles to be acute with sum less than \( \frac\pi2 \).
2. Use the tangent of the sum of the two acute angles.

Answer: (a) values less than 0 or greater than 2

Explanation:
1. Simplify: \( y=\frac{(\cos\theta-1)(\cos\theta+1)}{\cos\theta(\cos\theta+1)}=\frac{\cos\theta-1}{\cos\theta}=1-\sec\theta \), with \( \cos\theta\ne0,\pm1 \).
2. Because \( \theta\ne n\pi \), \( |\cos\theta|<1 \), so \( |\sec\theta|>1 \).
3. If \( \sec\theta>1 \) then \( y<0 \); if \( \sec\theta<-1 \) then \( y>2 \). So \( y<0 \) or \( y>2 \).

Teacher's Note:
1. Factor the numerator \( \cos^2\theta-1=(\cos\theta-1)(\cos\theta+1) \) and cancel \( \cos\theta+1 \) (valid since \( \theta\ne n\pi \)).
2. \( y=1-\sec\theta \) is a simple expression to analyse.

Answer: (a) \( (0,1] \)

Explanation:
1. Let \( x=\tan A \). Then \( \tan B=\tan\left(\frac\pi3-A\right)=\frac{\sqrt3-x}{1+\sqrt3x} \), and \( y=\tan A\tan B=\frac{x(\sqrt3-x)}{1+\sqrt3x} \).
2. Since \( 0<A,B<\frac\pi3 \), \( x\in(0,\sqrt3) \). The equation \( y(1+\sqrt3x)=\sqrt3x-x^2 \) is \( x^2+\sqrt3x(y-1)+y=0 \); for real \( x \), \( 3(y-1)^2-4y\ge0 \), so \( y\le\frac13 \) or \( y\ge3 \). With \( 0<\tan A\tan B<3 \), we get \( 0<y\le\frac13 \).
3. Hence \( 3\tan A\tan B\in(0,1] \); the value 1 occurs at \( A=B=\frac\pi6 \).

Teacher's Note:
1. Use \( \tan\left(\frac\pi3-A\right) \) and require a real solution \( x \) for the resulting quadratic.
2. Use the bounds \( 0<\tan A,\tan B<\sqrt3 \) to reject \( y\ge3 \).