Purpose: To validate the accuracy of 3D ultrasound (US) images compared with CT and MRI images based on the fact that ultrasound system provides better delineation of soft tissue. We investigate the accuracy of volume measurement of 3D US system and assess the image quality of 3D fusion for CT‐US and MRI‐US. Method and Materials: A phantom study was performed to ensure the accuracy of ultrasound image compared with other modalities. We have developed phantom consisted of background material (2% agarose) and a target. 3D ultrasound, CT, and MRI scanned images of 8 prostate cancer patients and phantoms are used for volume measurement and image fusion. We contoured phantom and patients images to obtain each volume. We developed 3D image fusion software based on the Insight Toolkit (ITK) and GIFT (Generalised image fusion toolkit) for dual modality data fusion using C++ and MFC language and thus acquired fusion images using this software. Results: In the volume contouring, phantom study showed 3D ultrasound system had small errors of 0.01–2.76% in sphere and ellipsoid phantoms. CT and MRI had lager errors of 1.62–11.34% and 0.31–7.95%, respectively. In addition, it is more accurate than 2D contouring which has errors of 0.01–35%. As a result of testing for software using phantom and patient data, we found that ultrasound image is well registrated with image of another modality. The 3D US‐CT fusion of patients showed that ultrasound is better system as compared to CT images for the imaging of bladder or prostate. Therefore 3D US image fusion with another modality is significant in prostate cancer treatment. Conclusion: The result showed that the ultrasound image fusion would provide higher accuracy in treatment margin. This study indicates that 3D ultrasound image and its fusion can contribute to the improvement in radiotherapy accuracy for prostate cancer.