FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.
-
Upload
meredith-wilcox -
Category
Documents
-
view
215 -
download
0
Transcript of FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.
![Page 1: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/1.jpg)
![Page 2: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/2.jpg)
FLogging: Metric Collection on the High
SeasJoe Ludwig
Director of DevelopmentFlying Lab Software
![Page 3: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/3.jpg)
What is FLogging?
Server-side Metric collection Game events with parameters Server performance events
![Page 4: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/4.jpg)
Formal + Log =
FLog
![Page 5: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/5.jpg)
Design Goals
Designer goals Lots of different views on data Possible to derive context from data
Programmer goals Easy to add new flog types No global recompiles
![Page 6: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/6.jpg)
Data Collection
![Page 7: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/7.jpg)
Adding a new flog type
Add any new field types in XML Add the new flog type in XML Upgrade database to generate SQL
Tables Procedures Views
Add call to record flog
![Page 8: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/8.jpg)
Flog types defined in XML
Flogger_field_types.xml:<flsFlogFieldType>
<name>level</name> <sqlType>TINYINT</sqlType>
</flsFlogFieldType>
![Page 9: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/9.jpg)
Flog types defined in XML
Flogger_types.xml:<flsFlogType>
<name>level_gained</name> <fieldNames>
<e>level</e> </fieldNames><usesObjectId>true</usesObjectId><hideByDefault>false</hideByDefault>
</flsFlogType>
![Page 10: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/10.jpg)
Add call to record flog
void flsAdvancementManager::setLevel(flsCharacter* pCharacter, igChar iNewLevel)
{ pCharacter->assignNewLevel(iNewLevel);
if(pCharacter->isPlayer()){
flsFlogger::flogf("level_gained", pCharacter->getLocalId(), "%d", iNewLevel);
}}
![Page 11: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/11.jpg)
Database Format - Tables
Flog table – one entry per flog Value tables – One entry per flog
that uses that field type Flog_type table – One entry per type
![Page 12: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/12.jpg)
flog table
Field Type
Id BIGINT IDENTITY
Flog_type INT
Flog_time DATETIME
Cluster_id INT
Character_id
INT
Puid BIGINT
![Page 13: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/13.jpg)
value table
Field Type
Id BIGINT
Value TINYINT
Type of value column is controlled by flogger_field_types.xml
![Page 14: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/14.jpg)
flog_type table
Field Type
Flog_type INT
Name CHAR(32)
hideByDefault
TINYINT
![Page 15: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/15.jpg)
Database Format – Stored Procedures Flog type procedures Flog type expiration procedures Flog_heartbeat procedure
![Page 16: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/16.jpg)
Flog type procedures
CREATE PROCEDURE Flog_level_gained @iObjectId INT, @level_param TINYINT
AS BEGIN DECLARE @pid bigint, @cid int;SELECT @pid = puid, @cid = clusterid FROM #flog_vars;
INSERT INTO flog (puid, cluster_id, flog_type, object_id) VALUES (@pid, @cid, 3, @iObjectId);DECLARE @lastid BIGINT; SET @lastid = SCOPE_IDENTITY();
INSERT INTO flog_level (id, value) VALUES (@lastid, @level_param);
END
![Page 17: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/17.jpg)
Flog age procedures
CREATE PROCEDURE Flog_Age_skill_used
AS DELETE flog FROM flog, flog_item_id WHERE DATEDIFF(DAY, flog.flog_time, CURRENT_TIMESTAMP)>=2 AND flog.id=flog_item_id.id
![Page 18: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/18.jpg)
Flog Heartbeat procedure
CREATE PROCEDURE Flog_Maintenance AS BEGIN
EXEC flog_Age_maneuver_skill_used; EXEC flog_Age_skill_used; EXEC flog_Age_volley_fired;
END
![Page 19: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/19.jpg)
Database Format – Type ViewCREATE VIEW Flog_Data_level_gained AS
SELECT flog.id, flog_level.value AS [level] FROM flog INNER JOIN flog_level ON flog.id=flog_level.id WHERE flog.flog_type=3
![Page 20: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/20.jpg)
What we flog
Login/Logout events Zoning Economic events Advancement events Missions Conquest events Chat GM commands
![Page 21: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/21.jpg)
![Page 22: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/22.jpg)
How much data?
4.5 million new flog records per week
Top 5 flog types account for 50%room_entered 18.95%xp_kill 11.78%exit_used 7.66%money_gained_kill 6.59%mission_stage_advanced 5.76%money_lost_recipe 4.20%
![Page 23: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/23.jpg)
![Page 24: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/24.jpg)
![Page 25: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/25.jpg)
![Page 26: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/26.jpg)
Example flog output
Time Type
7/10/2007 2:59:17 PM
chat_society message: ahoy Joany View Log
7/10/2007 2:31:38 PM
room_entered room: Main zone: P_Br_WstEnd
zone_context:
View Log
7/10/2007 2:31:35 PM
rpc_priv account: 1942 name: ZoneToPlayer
View Log
7/10/2007 2:31:27 PM
rpc_priv account: 1942 name: SendSystemChat
View Log
7/10/2007 2:31:27 PM
chat_system message: Ahoy! I'm your duty GM. If you are experiencing problems, feel free to send me a tell
(/tell GM Rhaegar, message). If you need to report a bug, please do so via
http://support.flyinglab.com
View Log
7/10/2007 2:27:47 PM
rpc_priv account: 1942 name: RequestCharacterInfo
View Log
7/10/2007 2:26:40 PM
chat_direct partner: 1042 message: Hi there - the best place to find info like
that is Nation chat
View Log
7/10/2007 2:16:55 PM
rpc_priv account: 1942 name: ZoneToPlayer
View Log
7/10/2007 2:15:11 PM
rpc_priv account: 1942 name: ZoneToPlayer
View Log
7/10/2007 2:14:48 PM
rpc_priv account: 1942 name: SystemWho
View Log
![Page 27: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/27.jpg)
![Page 28: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/28.jpg)
Money GainedReason Amount
money_gained_mission 31,310,500
money_gained_trade 11,038,331
money_gained_kill 8,029,993
money_gained_auction 6,826,344
money_gained_shop 4,123,899
money_gained_create_society 13,000
money_gained_respec 1,263
money_gained_mail 350
![Page 29: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/29.jpg)
Money LostReason Amount
money_lost_trade 11,038,331
money_lost_recipe 10,980,920
money_lost_auction 9,956,249
money_lost_shop 7,587,867
money_lost_ActivateStructureDeed 4,760,800
money_lost_admin 3,338,500
money_lost_recipe_tax 633,113
money_lost_PayStructureUpkeep 510,225
money_lost_auction_listing_fee 159,867
money_lost_create_society 89,000
money_lost_mail 14,550
money_lost_AuctionRelist 1,566
![Page 30: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/30.jpg)
Exploit Detection
Character XP Velocity Date10589 10,713 8/20/200710589 10,098 8/19/200711268 9,097 8/17/2007515 8,370 8/18/200711077 8,316 8/19/20077237 8,226 8/18/20073344 7,522 8/20/20076801 7,275 8/17/2007310 7,063 8/21/2007
![Page 31: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/31.jpg)
Reports using context
Context is available in the data Extracting it can be slow Example: Deaths per level
![Page 32: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/32.jpg)
Current level functionCREATE FUNCTION level_at_flog (@iObject INT, @iCluster INT,
@iFlogId BIGINT)RETURNS INT AS BEGIN
DECLARE @iLevel INTSELECT TOP 1 @iLevel = flog_level.valueFROM flog INNER JOIN flog_level ON flog.id=flog_level.idWHERE (flog.id < @iFlogID) AND
flog.object_id=@iObject AND flog.cluster_id = @iCluster AND
flog.flog_type=3ORDER BY flog.id DESC
IF @iLevel IS NULL SET @iLevel=1RETURN @iLevel
END
![Page 33: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/33.jpg)
Death by level table
CREATE TABLE death_by_level(id BIGINT,level INT,object_id INT,cluster_id INT)
![Page 34: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/34.jpg)
Death by level table
INSERT INTO death_by_level (id, level, object_id, cluster_id)
SELECT flog.id, dbo.level_at_flog(flog.object_id, flog.cluster_id, flog.id) AS [Level],flog.object_id, flog.cluster_id
FROM flog WHERE flog.flog_type=65
![Page 35: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/35.jpg)
Death by level table
id Level object_id cluster_id130804 7 1 0199407 3 45 0231699 7 41 0242162 4 43 0348568 8 74 0382145 7 75 0384861 8 86 0390520 8 86 0402353 9 22 0423383 9 33 0
![Page 36: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/36.jpg)
Death by level report
SELECT level, COUNT(id) AS [Deaths]
FROM death_by_levelGROUP BY levelORDER BY level
![Page 37: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/37.jpg)
![Page 38: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/38.jpg)
![Page 39: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/39.jpg)
More data from context
Death by zone Death by mission XP by zone Skills purchased by level Play time per level
![Page 40: FLogging: Metric Collection on the High Seas Joe Ludwig Director of Development Flying Lab Software.](https://reader036.fdocuments.net/reader036/viewer/2022070407/56649e2f5503460f94b1f1a1/html5/thumbnails/40.jpg)
Room to improve
More and better reports Automatic data rollups Parameterized queries Multiple associated fields in one
table Performance