Studies of new materials with low mechanical and optical losses for the coatings of future gravitational wave detectors