Yes, you need to give the refinery the Oil, you also *need* to give the refinery access to a city. It will only produce if there is a demand for it from a city.
The order is the opposite of the way people think… A Factory will only start to produce if it has a decent link to a City to give it’s goods. A “mine” (ie, oil) will only give the raw material if the factory it has a decent link to is producing.
Most people would think the other way, you deliver goods to a factory that it blindly turns into goods and then you pass it on to a city.