In data-scarce situations, causal discovery (CD) algorithms often produce unreliable causal relationships that may conflict with expert knowledge, especially in the presence of latent confounders. Additionally, most CD methods lack adequate uncertainty quantification, hindering users' ability to evaluate and refine results. To address these issues, we present a fully probabilistic CD method referred to as Ancestral GFlowNets (AGFNs). In a nutshell, AGFNs sample ancestral graphs (AGs) proportionally to a score-based belief distribution, allowing users to assess the uncertainty of the discovered causal relationships. On top of that, we design an elicitation framework that enables the incorporation of human knowledge into the inference process via importance sampling. Notably, our approach naturally accommodates CD on data sets with latent confounding and potentially heterogeneous data types, a setting that has received little attention from the literature. Finally, experimental results with observational data show that our method effectively samples from distributions over AGs and significantly enhances inference quality with human aid.