Just like any other wardrobe capsule, think basics first. Key pieces in neutral colours are a great place to start and will make it easy to build outfits that can take you to more places than just the gym. Begin with a well fitting pair of black compression capris/tights and a pair of joggers, a couple of black sports bras and 3 longer length tops. These basics will work together and with other items to create many different outfits to suit whatever your day throws at you. Obviously, the more times a week your workout, the more pieces you will need to invest in (or do the laundry more often!).
Sometimes it is better to pay a little bit more for performance fabrics. Fabric that is moisture wicking, quick drying and odor resistant is the key to being able to go from the gym to a coffee shop without feeling icky.