Since the discovery of superconductivity in 1911, the journey towards the realization of a room temperature superconductor and the understanding of the nature of unconventional superconductivity is still ongoing up to date. Despite tremendous theoretical and experimental efforts, the regular emerging of new unconventional superconducting materials provides regularly new challenges to describe their unknown mechanisms and novel phenomena. Nevertheless, many researchers are continuously trying to overcome the high mountain. Among them, uncovering the mechanism of unconventional superconductivity is of primary interest. In this thesis work, exotic states and pairing gap symmetry are studied in newly discovered unconventional superconductors. First, we investigate the re-entrant tetragonal phase in the Fe-based superconductor Ba0:76K0:24Fe2As2 by thermodynamic and thermoelectrical measurements. The reversible DC magnetization confirms by a thermodynamic method that the spin alignment in the re-entrant C4 phase is out-of-plane, in agreement with an itinerant double-Q magnetic order. The Nernst coefficient shows the typical unusually large negative value in the stripe-type spin density wave (SDW) state owing to the Fermi surface reconstruction associated with SDWand nematic order. At the transition into the re-entrant C4 tetragonal phase it hardly changes, which indicates that instead of a complete vanishing of the charge order, the spin reorientation triggers a redistribution of the charges to form a secondary charge order, e.g. in form of a chequerboard-like pattern that no longer breaks the rotational symmetry. Second, the magnetic phase diagram near the upper critical field is studied in the KF2As2 superconductor by resistance, magnetic torque and specific heat experiments using a high-resolution piezorotary positioner to precisely control the parallel alignment of the applied magnetic field with respect to the FeAs layers. We observe a clear double transition when the field is strictly aligned in the plane and a characteristic upturn of the upper critical field line, which goes far beyond the Pauli limit at 4.8 T. This provides firm evidence that a Fulde-Ferrell-Ovchinnikov (FFLO) state exists in this iron-based KFe2As2 superconductor. Third, we study the gap symmetry of the newly discovered topological superconductor NbxBi2Se3 by DC magnetization, specific heat and thermal expansion experiments. We find a clear two-fold nematic superconducting state in field-angle-resolved experiments including DC magnetization in combination with magnetoresistance data. We find that broad maxima and minima appear in the in-plane angular dependence of the upper critical field, which vary among different samples, which may be due to the nematic multi-domain effect. By measuring the field-angle resolved heat, we further confirm the existence of multiple nematic-domains. In addition, we observe the highly coupled structural distortion that coincides with the superconducting transition in high-resolution thermal expansion experiments. Finally, the pairing mechanism and gap symmetry in the Ising superconductor NbSe2 is examined with field-angle resolved magnetoresistance experiments. We observe a mixture of a six-and two-fold angular dependence of the upper critical field within the plane in a monolayer NbSe2, which is inconsistent with the crystal lattice symmetry. This provides firm evidence that novel superconducting are induced by a strong in-plane magnetic field. We experimentally confirm the existence of two distinct topological superconducting phases, which will need further studies and stimulate intensive theoretical works.