Stir olive oil, garlic, rosemary, and oregano together in a small bowl and let sit at room temperature for at least 15 minutes for the flavors to infuse.

Preheat a grill to medium high.

Place potatoes in a pot of salted water and bring to a boil. Reduce heat to a simmer and cook until potatoes are tender when pierced with the tip of a knife, about 12 minutes. Drain potatoes, discarding water.

Using a towel wrapped around the palm of your hand, gently smash each potato until the skin breaks, while trying to keep the potato whole (leaving it about 1 inch thick). Transfer potatoes to a baking sheet. Generously brush both sides of potatoes with olive oil mixture and season both sides with salt and pepper.

Place potatoes on grill and cook until grill marks appear and potatoes are nicely caramelized, 3 to 4 minutes per side. Transfer to a serving platter and crumble feta over the top. Serve hot or warm, drizzled with a bit of olive oil.