Objective To investigate the biotransformation of dehydrocostlactone in the liquid fermentation of endogenic fungus Y0. Methods The endophytic fungus of Y0 were purified and cultured, and then the transformed products were isolated and identified by chromatographic and spectral techniques. The toxicity to SW620 cells was evaluated by MTT assay. The targets information of two compounds and colon cancer SW620 cells was obtained by using online pharmacologic database. Further molecular docking of the compounds with targets with strong binding energy of colon cancer SW620 cells was performed. Results Two compounds were obtained by the way of biotransformation from dehydrocostlactone in endogenic fungus Y0 and identified as (1S, 3S, 5S, 6S, 7S, 11S)-3-hydroxl-11, 13dihydrodehydrocostuslactone and(1S, 3S, 5S, 6S, 7S, 11R)-3-hydroxyl-11, 13-dihydrodehydrocostusl actone. The IC50 of compounds 1 and 2 against coloncancer cell SW620 were(42.82 ± 8.32) μg/mL and(29.23 ± 7.17) μg/mL. Compounds 1 and 2 have binding targets to colon cancer SW620 cells. Conclusion Compounds 1 and 2 are guaiacane sesquiterpenoids. Compound 1 was more cytotoxic to SW620 cells than compound 2.