Você está certo de que não sabemos exatamente quando ele foi embora:
No entanto Celeborn provavelmente ficou para garantir o sucesso de seu novo reino élfico, East Lorien. Os elfos também só saíram quando estavam cansados da Terra Média. No caso de Celeborn e Galadriel, parece que a esposa estava cansada diante do marido, embora não fosse incomum que os parceiros élficos passassem anos separados (ver Elrond e Celebrían).It is said that Celeborn went to dwell [in Rivendell] after the departure of Galadriel; but there is no record of the day when at last he sought the Grey Havens, and with him went the last living memory of the Elder Days in Middle-earth.
Celeborn and Thranduil met in the midst of the forest; and they renamed Mirkwood Eryn Lasgalen, The Wood of Greenleaves. Thranduil took all the northern region as far as the mountains that rise in the forest for his realm; and Celeborn took the southern wood below the Narrows, and named it East Lórien; all the wide forest between was given to the Beornings and the Woodmen. But after the passing of Galadriel in a few years Celeborn grew weary of his realm and went to Imladris to dwell with the sons of Elrond. In the Greenwood the Silvan Elves remained untroubled, but in Lórien there lingered sadly only a few of its former people, and there was no longer light or song in Caras Galadhon.
Todas as citações dos Anexos.