Start by sterilizing any mason jars and lids before using. Place the in a big pot of boiling water for about half an hour or run them through the dishwasher. Place the peppers into the jars.
Combine the white vinegar or apple cider vinegar, water, kosher salt, and sugar in a large pot and bring to a boil. Remove from heat.
Divide the vinegar solution between the four jars, leaving about 1/2 inch of headspace and top with about one tablespoon of olive oil. Seal the lids and process the jars for 10-15 minutes in a hot water bath. Let the pickled peppers sit for a few weeks before diving in. After opening, refrigerate the jars.