• News
• Articles
• Projects
• Code
• Books
• About

Half Life 2 - Black Mesa Modification - Networking Part 1

Arne Olav Hallingstad
9th December 2008
Updated 11th December 2008
Home

Black Mesa is a mod that recreates the single player Half Life experience in the Half Life 2 source engine, with updated levels, textures, models, sounds and graphics. The gameplay experience is kept the same as much as possible. There will be support for co-op in Single Player, but there are several technical challenges that needs to be overcome before players can fully enjoy the co-op experience.

Client-Server Networking Model

Half Life 2 uses a client-server networking model where the client code is in a different DLL than the server code. This hasn't changed since the Quake series games which did the same. Doom 3 merged the client/server code into a single gamecode DLL, this made it easier to guarantee the client and server ran the same code and that clients would predict correctly.

Clients run behind the server and will only interpolate between snapshots, if it runs out of snapshots it starts to extrapolate from the most recent snapshots.

IClientNetworkable interface used for synchronizing entities.

debugoverlay->AddLineOverlay for debugging

Console Variables

• cl_interpolate - Disable/enable interpolation on clients
• cl_lagcompensation - VDC: Failing to turn off lag compensation at the same time as interpolation on can lead to undesired results.
• maxplayers - Maximum number of clients connected to a server.
• net_fakelag - Fake network lag, together with maxplayers > 1 you can run client/server on the same machine by spawning on a map
• cl_showerror - will log prediction errors

Network Synchronization of func_movelinear

func_movelinear is an entity used a lot in single player, but not at all in multiplayer (as far as I'm aware). It's used for allowing basic platforms/lifts to move linearly between two states.

Several tests have been performed on the client side using a box map with a single func_moverlinear platform moving between two positions at a velocity of 100 units per second. Setting cvar maxplayers to 2, and net_fakelag to 30. Playing as a listen server the constant motion of the platform is nice and smooth, playing with some lag in a multiplayer game will cause the platform to jitter and it doesn't look very good, something that should not happen since the client should interpolate properly between snapshots.

Server

• Physics simulation happens in CBaseEntity::PhysicsSimulate
• func_movelinear uses MOVETYPE_PUSH which does the actual movement in CPhysicsPushedEntities::PerformLinearPush.

Client

All entities should interpolate between snapshots since the client is always running behind the server. This makes prediction very easy, if client runs out of snapshots it needs extrapolate into the future. The only exception is the local player which should run prediction in the future as local player input needs to be immediate.

• C_BaseEntity::PostDataUpdate/PreDataUpdate - Set the last/current snapshot
• func_movelinear uses MOVETYPE_PUSH which does the actual movement in CPhysicsPushedEntities::PerformLinearPush.
• OnRenderStart - Does the actual interpolation of all server entities, func_movelinear as well

The origin of interpolated entities on clients are updated like this:

1. A new snapshot is received from the server and the engine calls RecvProxy_VectorToVector to update the m_vecNetworkOrigin vector.
2. C_BaseEntity::PreDataUpdate calls Interp_RestoreToLastNetworked() if needed.
3. C_BaseEntity::PostDataUpdate calls MoveToLastReceivedPosition() which sets the origin to the newly set m_vecNetworkOrigin value. The function might be called multiple if many snapshots have been received, framecount will increase for each call to this function.
4. OnRenderStart() does all entity interpolation after snapshots have been set. C_BaseEntity::Interpolate is being called.

Tests

The yellow line is the client position of the mover over time. You can see it's position moves smoothly which is what we would expect from an interpolation function implemented correctly. If you instead look at it's position from frame to frame, the red line, it is jagged. Though the red line is only useful for illustrating that interpolation was done at irregular times.

x-axis: Time since start of the map.

y-axis: Y position of platform
yellow line: interpolated origin (ie. it's position as seen from the client)
red line: it's position at each frame. Since frames per second changes this is not the same as the yellow line.

The velocity of the platform. It just confirms that the interpolation is correct. Though it should not have spiked in either direction as the platform changes direction at frames ~450 and frames ~900 unless it did extrapolation, which i doubt it was doing. So this is something to look into later once the jerky movement has been fixed.

x-axis: Frame number

y-axis: Velocity in units per second
blue line: Velocity of the platform on the client.

FPS on the client. This is interesting to note because I'm quite sure these spikes corelate to the instances of jerky movement. The client regularly drops to 10 fps over a couple of frames.

x-axis: Frame number

y-axis: Frames per second on the client
blue line: Frames Per Second

The change in position of the platform from one frame to the next.

x-axis: Frame number

y-axis: Change since the last frame in units.
blue line: Frames Per Second

All data was logged right after the interpolated origin was set.

The jerky movement happened as the client FPS dropped, the server FPS suspiciously dropped at the same time usually, so there would be a longer than usual delay before client received the next snapshot. This shouldn't have been an issue though since the client did usually not go into extrapolation anyway. It could be that interpolated entities are not interpolated as often as the local player and the renderer. It did feel smooth moving around, it was just the platform that was jerky.

As a note, the platform physics on the client is ONLY being set when interpolating between snapshots (and temporarily set when recieving new snapshots). The client is not running physics on it or anything.

Possible Reasons For Jitter

1. When the client receives new snapshots, it sets the origin back to the last snapshot. It is then supposed to interpolate the origin to get the proper position before rendering the entity. Is it resetting the origin, but not interpolating before rendering? Theory testable: Log the origin at the last possible time before the gamecode returns and entities are rendered (for graphs above the origin logged was right after interpolating, the origin could be modified before rendering).
2. The client does for some reason not interpolate the platform origin every render frame. But the player origin is predicted every frame so moving around feels smooth. If this is correct then ALL interpolated entities using C_BaseEntity's interpolation is not updated correctly. Find it hard to believe there's a problem with player interpolation though so the fix could be in the difference between how C_BaseEntity does interpolation and how the player does interpolation (C_BaseHLPlayer?). Theory testable: By logging all updates of the platform and player origin.
3. Does the server send a full snapshot for the movign platform instead of a delta compressed one and this would explain the sudden drop in FPS on both server and on the client decoding and predicting the new position? C_BaseEntity::PostDataUpdate could be called many times if several snapshots are in the client queue. Theory testable: See if there is a spike in snapshots decoded happening close to the drop in FPS.
4. An FPS drop itself is the reason: Drops to ~10FPS for ~2 frames, happens at regular intervals. Now what causes the FPS drop and why at so seemingly regular intervals? Profile server and client code, but this is a shot in the dark, might not be woth the time. FPS drop could probably only be fixed in the engine. Theory testable: Profile interpolation and all the other major gamecode portions and see if there's spikes.
5. Instead of not being updated as often as it should, interpolated entities might not have proper gpGlobals->curtime, but the players interpolated, and local player do? I find this unlikely though, hmm. Theory testable: Print interpolation time and amount to interpolate from last snapshot and compare the platform with a player, not local player as local player is forward predicted.

And If Everything Fails...

If this will be impossible to fix because the problem lies in the engine then there is only one option left; replace these server only entities with server/client entities. This might increase network traffic as clients no longer just interpolate, but must run their physics and receive required state change information. The physics for these are deterministic and have very simple state changes though, so perhaps it decrease the network traffic but require more CPU on clients. (thinking of func_movelinear, func_tracktrain, func_door, func_platrot, etc.).

TODO: is the local player origin updated more often (from local player prediction) than the platform origin (from interpolation), this could explain smooth local player movement but jerky platform movement.

TODO: Profile server and client code to see if the gamecode causes the delay? If not then bugger, might need to work around this problem if so.

TODO: Testing multiplayer using a single PC and one instance of HL2 isn't optimal. Second PC should run the server to see if any behavior is different, if not then it's probably ok to continue to debug multiplayer on one PC.

Facts

• glGlobals contains loads of interesting values related to ticks, frames, frametime, etc.
• Check entity name with FClassnameIs( pCur, "class C_BaseEntity" ). Entities spawned on the server only has the C_BaseEntity class on clients which makes it difficult to distinguish two func_moverlinear platforms on a client for example.

Questions

• Is it possible to get the current frame number on the client? Yes: gpGlobals->framecount
• How can you tell the name of an entity on the client? No, but you have FClassnameIs( pCur, "class C_BaseEntity" )?

Part 2