Velocity based on the number of story points a team has historically delivered is certainly not random guessing.

During spring planning, velocity should be used as a reality check to prevent a team that has historically delivered 20 points per sprint from optimistically (and unrealistically) taking on 30 points, 40 points, or more and/or as a data-based argument to prevent business/product owners from pressuring the team to take on more than it can actually deliver.

Of course all this is based on consistent story point estimation, challenging estimates during backlog grooming, etc.