The spin correlations in a series of cobaltate polycrystalline samples with Swedenborgite structure, ABaCo₃BO₇ (A = Y, Ca, and B = Co, Fe, Al, Zn), were studied by means of diffuse polarized neutron scattering. The alternate stacking of kagome and triangular Co layers forms a tetrahedral Co-network with geometrical frustration of antiferromagnetically (AF) coupled spins, which typically suppresses long-range order even at low temperatures (~1K) despite Curie-Weiss temperatures in the order of 1000 K. The generic diffuse magnetic scattering as found in Y_0.5Ca_0.5BaCo₄O₇ reveals two dimensional (2D) spin correlations on the kagome sublattices towards the entropically favoured × structure and suggests a decoupling of layers by low spin states S = 0 on triangular sites. Co-substitution by Al and Zn yields similar diffuse magnetic scattering, however, spin dilution results in even more disordered spin liquid or spin glass states. With B = Fe or Co, differences in the magnetic scattering evolve, indicating the onset of spin correlations perpendicular to the kagome layers and non-vanishing spin states on the triangular sites. Geometrical frustration prohibits any 3D long-range order when the characteristic 2D spin correlations with the × motif is present. We observe long-range order only in YBaCo₄O₇ and CaBaCo₃FeO₇ showing rather distinct ordering wave-vectors.