ENEDIS (formerly called ERDF) is responsible for the electricity network and can link up the house for you. The alternative is to go via one of the energy companies (EDF, Direct Energie etc) which can act as an intermediary (in any case you would need a contract with a supplier before you can be connected).
Linking up a property to the network is called raccordement and, yes, there is a cost, which varies depending on the complexity of the site. If you supply them with details, they can provide a devis (estimate).
Factors affecting the price include whether underground or overground lines are used, geographical location and amount of power needed (the average French home uses 6 kVA, but large homes with electric heating may need 9-12 kVA).
In some cases Enedis may need to undertake an extension de réseau (extension of the network), bringing the electricity distribution network close enough to the home for it to be linked up. If applicable the local council may be able to pay for that part of the costs.
An Enedis spokeswoman said it is not possible to estimate the costs as there are too many variables, however it will be higher if the home is far from the public highway.
You can enter your details and obtain an estimate here: connect-racco.enedis.fr/prac-internet/login