The recent global dissemination of Chikungunya and Zika has fostered public health concern worldwide. To better understand the drivers of transmission of these two arboviral diseases, we propose a joint analysis of Chikungunya and Zika epidemics in the same territories, taking into account the common epidemiological features of the epidemics: transmitted by the same vector, in the same environments, and observed by the same surveillance systems. We analyse eighteen outbreaks in French Polynesia and the French West Indies using a hierarchical time-dependent SIR model accounting for the effect of virus, location and weather on transmission, and based on a disease specific serial interval. We show that Chikungunya and Zika have similar transmission potential in the same territories (transmissibility ratio between Zika and Chikungunya of 1.04 [95% credible interval: 0.97; 1.13]), but that detection and reporting rates were different (around 19% for Zika and 40% for Chikungunya). Temperature variations between 22°C and 29°C did not alter transmission, but increased precipitation showed a dual effect, first reducing transmission after a two-week delay, then increasing it around five weeks later. The present study provides valuable information for risk assessment and introduces a modelling framework for the comparative analysis of arboviral infections that can be extended to other viruses and territories.