Can you wear the coyote tan/brown undershirt with civilians?

In short, yes.

Here's the longer answer:

According to AR 670-1, section 3-9 states, "Uniform items authorized for wear with civilian clothing by males are restricted to the gold cuff links, studs, tie bar, mourning band, footwear, socks, gloves, undergarments, black bow-tie, wool scarf, all-weather coat, fleece caps, and physical training uniforms." The key word in this section is "undergarments".

Later on in AR 670-1, the regulation refers you to DA Pam 670-1 regarding what items constitute "undergarments".

In DA Pam 670-1, section 18-30, the tan undershirt is listed as an undergarment.

"e. Undershirt, sand or tan 499 (male and female).
(1) Type. The undershirt is a clothing bag issue item or optional purchase items.
(2) Description. The material is cotton knitted cloth or 50 percent cotton and 50 percent polyester, with quarter- length sleeves and a crew neck, or is of a similar commercial design.
(3) How worn. All personnel will wear the sand or tan 499 undershirt with all utility uniforms, except for and garrison culinary uniforms."

So, if DA Pam 670-1 defines the undershirt as an undergarment (which it does), and AR 670-1 uses that definition (which it does) and says that wearing undergarments with civilian clothing is allowed (which it also does), then you are good.

Why would you want to? lol

The AR for Wear and Appearance of Army Uniforms is AR 670-1 so that would be the first place I'd check if I ever had a doubt about uniforms or appearance.

e. Wearing a combination of civilian and military clothing is prohibited, unless prescribed in this regulation or directed by the Secretary of the Army.

g. Uniform items authorized for wear with civilian clothing by males are restricted to the gold cuff links, studs, tie bar, mourning band, footwear, socks, gloves, undergarments, black bow-tie, wool scarf, all-weather coat, fleece caps, and physical training uniforms.
i. Uniform items that have been declared obsolete may be worn with civilian clothing, provided such items contain no distinctive insignia or buttons.

Based on 670-1 no you can't wear the brown shirt with civilian clothing. It specifically states what uniform items are authorized with civilian clothing. I don't see tan or brown shirts in there.