There are a lot of anachronistic events at the Olympics, like the modern pentathlon, that you assume are just grandfathered into the Olympics at this point. But how does a sport like golf wind up debuting at the 2016 Rio Olympics? It's actually an involved process to add a sport to the Olympic Games.
To start, the sport has to have a non-governmental international federation. Traditionally, that body can then petition the International Olympic Committee (IOC) for Olympic inclusion. However, in 2014 the IOC formalized another avenue that created a more flexible Olympic program to make it easier for events like skateboarding and surfing to be added, as they will for the 2020 Tokyo Olympics. That will be the first Olympics to have sports who have taken advantage of that new procedure. Golf had to take the more traditionally used route.
Britannica notes that a sport can be admitted "as a sport, a discipline, which is a branch of a sport, or an event, which is a competition within a discipline. For instance, triathlon was admitted as a sport, debuting at the 2000 Games in Sydney. Women's wrestling was a new discipline in the sport of wrestling at the Athens Games, and women's pole vaulting was the most recently added track and field event."
The actual process begins with the international federation filling out a questionnaire that can be in excess of 100 pages. It asks questions about gender equity, how many countries are involved in the sport, and how large the global audience is for the sport. That last point is measured by TV audience, social media, and event attendance. Another consideration is the cost of building venues to hold the event and availability of those venues in upcoming host cities.
Traditional guidelines stated that a sport must be practiced by men in at least 75 countries on four continents, and in 40 countries on at least three continents for a women's event. There is also a ban on anything that is purely a "mind sport," like chess, or a sport that relies on mechanical propulsion, like automobile racing.
The IOC also weighs the appeal of the proposed sports to younger generations. That's in part because Olympic sponsors want to target that 18-34 age range and having an event like BMX biking helps sponsors get there.
"We want to take sport to the youth. With the many options that young people have, we cannot expect any more that they will come automatically to us. We have to go to them,” said Thomas Bach, IOC President, in a press release announcing the new games for Tokyo.
Let's take a stab in the dark and say youth-appeal isn't why golf wound up at Rio.