Every province in the world produces something — timber, wheat, fish (some coastal provinces), coal, iron, etc. And every country needs to consume things — maybe they need wheat to supplement their diet of fish, or maybe they need dyes in order to make their clothing or fabrics more colourful.
Therefore, there is a need for countries to trade with each other. In fact, it sustains everybody’s economy. Very few countries are entirely self-sufficient, even at the beginning of the game. But as countries acquire new technology, learn about different concepts of government, or modernise their militaries, they will eventually need to buy things from other countries, even if they could remain isolated in the early game.
All goods have value, but their price depends entirely on supply & demand — how many people want to buy the stuff, and how much of it there is to go around. If you’re talking about cars, or telephones, they’re not easy to come by during this time period. In fact, each of these items requires the factory to buy materials from other factories just so they can produce them! So you’re not just paying for the cost of transport, you’re paying for the cost of purchasing the component parts, and providing enough to give the producer a profit on top of it all.
Trade isn’t always “free” — in Victoria 2, different things may interfere. Sometimes it’s a government tariff on foreign imports, called a tariff. But during times of war, another country’s navy may also blockade its enemy’s seashore. This can dramatically increase the price of the goods traded, as smugglers may have to slip everything past the menacing warships!