Serato DJ Pro General Discussion

Talk about Serato DJ Pro, expansion packs and supported hardware

Stats for Serato

Djelad00 12:10 AM - 14 May, 2019
Hi,

I wrote an app that can process statistics from historical Serato session files. I'm opening up the beta in hopes of gathering feedback on how to best keep developing it.

Beta functionality is currently limited to 200 session files and only includes a couple basic stats (top 200 artists/songs), but I will be adding more soon.

The jar can be download from here (must have JRE 8 installed; only tested on OSX but may be windows compatible):
gitlab.com

I've also added some screenshots and explanation on how to use it here:
projects.ssldev.org

any and all feedback is welcomed!
Bobakiss 9:41 PM - 31 May, 2019
This is real nice! just ran it on High Sierra with no issues.
Djelad00 11:54 PM - 31 May, 2019
thanks! and thanks for letting me know..
The Return of Dj Sparky 8:58 PM - 6 June, 2019
i wonder if Serato jacked your concept on how you record the play count as they are conveniently adding it to 2.2....
Djelad00 4:19 AM - 7 June, 2019
lol doubt it.. but good to see it being added natively after all this time!
AKIEM 7:05 AM - 7 June, 2019
eh, Im going with jacked
deejdave 4:51 PM - 7 June, 2019
You would be wrong there
cotdagoo 6:05 PM - 7 June, 2019
Quote:
i wonder if Serato jacked your concept on how you record the play count as they are conveniently adding it to 2.2....

Serato's method only counts plays after the feature was added. Not a historical count based on your play history :(

So this is actually better than what Serato has delivered unfortunately.
Culprit 12:27 AM - 10 June, 2019
Dope bro I love off products helping us analyze data. If you could update nik's old scratchtools to support Serato DJ I'd throw some money your way bro.
dj_soo 3:05 AM - 11 June, 2019
How easy would it be to utilize this program to analyze history and then write previous played files to the new play count feature in serato?
dj_soo 3:06 AM - 11 June, 2019
This feature has also been requested since the scratchlive days. Highly doubt it was “jacked.”
Djelad00 5:51 AM - 11 June, 2019
it would depend where the play count is stored. if it's an id3 attribute, it wouldn't be too hard, but it's more likely to be persisted within Serato or the DB file.

having said that, it's not something I would implement. Serato could have easily done so, but chose not. I'm sure they had their reasons...

moreover, It's also not the direction I'm trying to take stats in. I thought it'd be more interesting to take a more retrospective look at the data dj's generate, beyond play count.

If there are any specific stat/trend/plots that any of you would find useful or interesting, I'd be happy to try to include them!
cotdagoo 12:06 PM - 11 June, 2019
Quote:
...
having said that, it's not something I would implement. Serato could have easily done so, but chose not. I'm sure they had their reasons...


Their reason was basically "they cant" - which doesn't make a whole lot of sense if 3rd party developers can get this data but the actual developers are unable to read a database and use the information inside to make a historical play count.

From staff -
Quote:

Unfortunately, we are unable to use the historical play data format to count towards play count so play count will start from this build.
AKIEM 11:04 PM - 11 June, 2019
Quote:

From staff -
Quote:
Unfortunately, we are unable to use the historical play data format to count towards play count so play count will start from this build.

where was this said?
deejdave 1:09 AM - 12 June, 2019
Quote:
Quote:
From staff -
Quote:
Unfortunately, we are unable to use the historical play data format to count towards play count so play count will start from this build.

where was this said?

It is a fact. Furthermore if I am not mistaken using an old build AFTER using the new build will reset it/mess it up ................ or something like that.
dj_soo 1:50 AM - 12 June, 2019
It won’t reset it, but it’ll blank out the field until you load up the track again.
The Return of Dj Sparky 6:08 AM - 12 June, 2019
Well if serato are quoted saying it's not possible and a user proves it's possible it just shows how shit the serato development team are now when a third party can do better work then they can
bumbo08 7:34 AM - 12 June, 2019
Quote:
Well if serato are quoted saying it's not possible and a user proves it's possible it just shows how shit the serato development team are now when a third party can do better work then they can

I don't think these are the same thing.... this app loads up the stats in a standalone app, not into Serato.

I may be wrong but the Serato quote was regarding using the play history to push the stats back into the library which is what they said they couldn't do.
dj_soo 10:41 AM - 12 June, 2019
Quote:
Well if serato are quoted saying it's not possible and a user proves it's possible it just shows how shit the serato development team are now when a third party can do better work then they can


two different things. Serato is saying it's impossible to write past history to the current play count - which this program doesn't do.
bumbo08 11:02 AM - 12 June, 2019
^ what I said :)
cotdagoo 1:31 PM - 12 June, 2019
Quote:
Quote:
From staff -
Quote:
Unfortunately, we are unable to use the historical play data format to count towards play count so play count will start from this build.

where was this said?

Was said to me in PM when I questioned why play count only started now, rather than counting my 10+ years of SSL/SDJ played history.

I don't see how starting counting from 0 now is useful to anyone really.. I don't care how many times I played a track since 2019.. I've been using this software for over 10 years lol..
cotdagoo 1:33 PM - 12 June, 2019
Personally I feel if I can see the data.. someone at Serato should be able to manipulate that data from whatever database it's stored in and extract relevant information (ie: historical play count)
AKIEM 5:44 PM - 12 June, 2019
Quote:
Quote:
Quote:
From staff -
Quote:
Unfortunately, we are unable to use the historical play data format to count towards play count so play count will start from this build.

where was this said?

Was said to me in PM when I questioned why play count only started now, rather than counting my 10+ years of SSL/SDJ played history.

I don't see how starting counting from 0 now is useful to anyone really.. I don't care how many times I played a track since 2019.. I've been using this software for over 10 years lol..
got it, yup.

smoke
Djelad00 2:08 AM - 24 August, 2019
Quote:
How easy would it be to utilize this program to analyze history and then write previous played files to the new play count feature in serato?


finally had a chance to check out the new Serato DJ tags.

It seems like Serato is storing the play count in the mp3, via the 'TXXX' ID3 field. I don't think it'll be too hard for me to write another program that would modify the field with the play counts that Stats has already collected.

I'll see if I can get something working this weekend...
cotdagoo 3:04 PM - 26 August, 2019
Quote:
Quote:
How easy would it be to utilize this program to analyze history and then write previous played files to the new play count feature in serato?


finally had a chance to check out the new Serato DJ tags.

It seems like Serato is storing the play count in the mp3, via the 'TXXX' ID3 field. I don't think it'll be too hard for me to write another program that would modify the field with the play counts that Stats has already collected.

I'll see if I can get something working this weekend...

This would be amazing! Still not getting much info/stats out of the official feature to make it useful in any way after a few months use.
Djelad00 2:20 PM - 6 October, 2019
okay I wrote a new tool that will scan all session files for counts and overwrite the serato TXXX:playcounts ID3 tag with the tabulated value.

couple of things:
1. the serato DB file (e.g. /Users/elad/Music/Serato/database V2) has to be deleted & rebuilt. for some reason overwritten count tags aren't showing up properly for me unless I force the DB to rebuild
2. because the DB file is deleted, some mp3s may be missing and will have to be re-imported into serato.

if you do decide to use this tool, please BACKUP your library & serato files before using it! Editing an mp3's ID3 tag is a destructive operations and may cause files to become corrupt. Use it at your own risk...

source code + instructions here:
gitlab.com

hope this helps some of you out :)
Djelad00 3:30 PM - 6 October, 2019
PS: I've only tested this tool against my library, running OSX Mojave... please let me know if it works for you
Culprit 5:27 PM - 6 October, 2019
Did you try rescanning your files in serato after the changes were made to autopopulate the count field?
dj_soo 5:28 PM - 6 October, 2019
Ooh, I’ll give this a try this week.
Djelad00 12:58 AM - 7 October, 2019
I did. Even so, serato would overwrite some counts with erroneous values or not show the values the tool wrote in unless the DB was deleted.

at some point I'll see if I can decode the DB file and figure out why. prob wouldn't be too hard to fix this, though re-adding missing mp3s and rebuilding the DB file wasn't that painful for me...
deejdave 2:35 AM - 7 October, 2019
Quote:
though re-adding missing mp3s and rebuilding the DB file wasn't that painful for me.

For those with massive libraries spanning YEARS of acquiring, editing and curating this may prove to be an impossible task. Not trying to take anything any from your work as this is the only item I have any discrepancy with.
Djelad00 4:55 AM - 8 October, 2019
Quote:
Quote:
though re-adding missing mp3s and rebuilding the DB file wasn't that painful for me.

For those with massive libraries spanning YEARS of acquiring, editing and curating this may prove to be an impossible task. Not trying to take anything any from your work as this is the only item I have any discrepancy with.


FWIW, as far as I can tell, the only info you lose by creating a new db file is the 'date added' info. Crates, loops, markers, etc, were not affected (not sure if this is true for video files)

support.serato.com
Culprit 6:39 PM - 8 October, 2019
i use my date added heavily
Djelad00 5:09 AM - 17 December, 2019
Think I've figured out how to modify the DB file in a way that preserves date added but overrides the count data. Hoping to have a new version out shortly...

meanwhile, I've added a year filter to the stats app which can show you your top songs/artists for a specific year. Kinda fun to see how they change over time.

comments/feedback welcomed!

screenshot:
projects.ssldev.org

download:
gitlab.com

source code:
gitlab.com
Djelad00 2:03 AM - 6 January, 2020
So turns out the serato DB file doesn’t need to be modified after all.

The “bug” seems to be with how serato reads in the play count tag. After some experimenting I finally managed to get serato to display the tag correctly by adding whitespaces around the count when writing it. Go figure…

I suppose the good news is that I now know how to programmatically overwrite and create serato files (e.g. crate, session, preferences, etc). I’ll add that capability to my API so it should hopefully open up opportunities to create other serato tools to manipulate those file types. If anyone has any interesting ideas feel free to hit me up.

So long story short it's NOT necessary to delete the serato DB file anymore. If you want to use the tool I’d recommend:
1. Shut down serato
2. Backup your files
3. Run the tool from terminal (you may need to install the JDK to run it)
4. Open serato and rescan all id3 tags

As always let me know if you run into any issues and I’ll do my best to fix them!
bumbo08 8:27 AM - 6 January, 2020
That's cool nice work!
cotdagoo 5:46 PM - 16 January, 2020
Thanks for doing all this work.

Been trying to get this going in Windows and coming up with a few errors when it starts up.

[12:44:19]ERROR SeratoSessionsLoaderService: [76794.session] failed to unmarshal [Index: 235, Size: 235]
(0 loaded, 6 more to go)
[12:44:19]ERROR SslSessionFileUnmarshalService: error encountered while unmarshalling [C:\Users\Andrew\Music\_Serato_\History\Sessions\50864.session]
java.lang.IndexOutOfBoundsException: Index: 1261, Size: 1261
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.ssldev.api.SeratoSessionFile.getLastAdatEndTimeOrDefault(SeratoSes
sionFile.java:59)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:52)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:34)
at org.ssldev.api.messages.SessionFileUnmarshalledMessage.<init>(SessionF
ileUnmarshalledMessage.java:28)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionRequest(SslSessionFileUnmarshalService.java:73)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
at java.util.concurrent.CountedCompleter.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

[12:44:19]ERROR SeratoSessionsLoaderService: [50864.session] failed to unmarshal [Index: 1261, Size: 1261]
(0 loaded, 5 more to go)
[12:44:19]ERROR SslSessionFileUnmarshalService: error encountered while unmarshalling [C:\Users\Andrew\Music\_Serato_\History\Sessions\76688.session]
java.lang.IndexOutOfBoundsException: Index: 103, Size: 103
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.ssldev.api.SeratoSessionFile.getLastAdatEndTimeOrDefault(SeratoSes
sionFile.java:59)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:52)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:34)
at org.ssldev.api.messages.SessionFileUnmarshalledMessage.<init>(SessionF
ileUnmarshalledMessage.java:28)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionRequest(SslSessionFileUnmarshalService.java:73)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
at java.util.concurrent.CountedCompleter.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

[12:44:19]ERROR SeratoSessionsLoaderService: [76688.session] failed to unmarshal [Index: 103, Size: 103]
(0 loaded, 4 more to go)
[12:44:19]ERROR SslSessionFileUnmarshalService: error encountered while unmarshalling [C:\Users\Andrew\Music\_Serato_\History\Sessions\81114.session]
java.lang.IndexOutOfBoundsException: Index: 316, Size: 316
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.ssldev.api.SeratoSessionFile.getLastAdatEndTimeOrDefault(SeratoSes
sionFile.java:59)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:52)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:34)
at org.ssldev.api.messages.SessionFileUnmarshalledMessage.<init>(SessionF
ileUnmarshalledMessage.java:28)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionRequest(SslSessionFileUnmarshalService.java:73)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
at java.util.concurrent.CountedCompleter.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

[12:44:19]ERROR SeratoSessionsLoaderService: [81114.session] failed to unmarshal [Index: 316, Size: 316]
(0 loaded, 3 more to go)
[12:44:19]ERROR SslSessionFileUnmarshalService: error encountered while unmarshalling [C:\Users\Andrew\Music\_Serato_\History\Sessions\61110.session]
java.lang.IndexOutOfBoundsException: Index: 377, Size: 377
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.ssldev.api.SeratoSessionFile.getLastAdatEndTimeOrDefault(SeratoSes
sionFile.java:59)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:52)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:34)
at org.ssldev.api.messages.SessionFileUnmarshalledMessage.<init>(SessionF
ileUnmarshalledMessage.java:28)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionRequest(SslSessionFileUnmarshalService.java:73)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
at java.util.concurrent.CountedCompleter.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.pollAndExecCC(Unknown Source)
at java.util.concurrent.ForkJoinPool.helpComplete(Unknown Source)
at java.util.concurrent.ForkJoinPool.externalHelpComplete(Unknown Source)
at java.util.concurrent.ForkJoinTask.externalAwaitDone(Unknown Source)
at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source)
at java.util.concurrent.ForkJoinTask.invoke(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source)
at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.util.stream.ReferencePipeline.forEach(Unknown Source)
at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionsRequest(SslSessionFileUnmarshalService.java:58)
at org.ssldev.core.mgmt.EventHub.lambda$add$2(EventHub.java:106)
at java.util.ArrayList.forEach(Unknown Source)
at org.ssldev.core.mgmt.EventHub.add(EventHub.java:106)
at org.ssldev.core.mgmt.AsyncEventHub.lambda$new$0(AsyncEventHub.java:27)
at org.ssldev.core.mgmt.AsyncEventHub$1.run(AsyncEventHub.java:45)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

[12:44:19]ERROR SeratoSessionsLoaderService: [61110.session] failed to unmarshal [Index: 377, Size: 377]
(0 loaded, 2 more to go)
[12:44:19]ERROR SslSessionFileUnmarshalService: error encountered while unmarshalling [C:\Users\Andrew\Music\_Serato_\History\Sessions\50458.session]
java.lang.IndexOutOfBoundsException: Index: 403, Size: 403
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.ssldev.api.SeratoSessionFile.getLastAdatEndTimeOrDefault(SeratoSes
sionFile.java:59)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:52)
at org.ssldev.api.SeratoSessionFile.<init>(SeratoSessionFile.java:34)
at org.ssldev.api.messages.SessionFileUnmarshalledMessage.<init>(SessionF
ileUnmarshalledMessage.java:28)
at org.ssldev.api.services.SslSessionFileUnmarshalService.handleUnmarshal
SessionRequest(SslSessionFileUnmarshalService.java:73)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
at java.util.concurrent.CountedCompleter.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

[12:44:19]ERROR SeratoSessionsLoaderService: [50458.session] failed to unmarshal [Index: 403, Size: 403]
(0 loaded, 1 more to go)
[12:44:19]WARN : THREAD32 - UnmarshalMultipleSessionsRequest execution slowness detected ( execution took 00:14)

Not sure if any of that makes sense, but happy to test anything if you're able to figure out what's happening :)
DJ Quartz 6:32 PM - 16 January, 2020
This is interesting....:)

I will definitely have to try this.

Good Work!
Djelad00 6:29 AM - 17 January, 2020
Quote:
Thanks for doing all this work.

Been trying to get this going in Windows and coming up with a few errors when it starts up.
...

[12:44:19]ERROR SeratoSessionsLoaderService: [50458.session] failed to unmarshal [Index: 403, Size: 403]
(0 loaded, 1 more to go)
[12:44:19]WARN : THREAD32 - UnmarshalMultipleSessionsRequest execution slowness detected ( execution took 00:14)

Not sure if any of that makes sense, but happy to test anything if you're able to figure out what's happening :)


thanks for letting me know! I think I fixed this issue a while back.. try downloading the latest Serato-play-counter from gitlab.com

if that doesn't work, do you mind sending me one of those session files? and I'll see if I can reproduce and fix on my end. I've only tested my code on OSX..
cotdagoo 3:59 PM - 17 January, 2020
Quote:

thanks for letting me know! I think I fixed this issue a while back.. try downloading the latest Serato-play-counter from gitlab.com

if that doesn't work, do you mind sending me one of those session files? and I'll see if I can reproduce and fix on my end. I've only tested my code on OSX..

Thanks for the quick reply!

Running it now, seems to be working with less errors/warnings. Opened a ticket on your gitlab to upload my session file and not clog things up here :)