oscam - a better BISS?

campag5242 · Sep 24, 2017

I've patched oscam-emu to perform better (for me at least) on feeds.

Background:
New to oscam & feeds, I soon became frustrated with not being able to zap between certain live feeds without having to edit the key files.

For feeds, oscam generates a fake ecm whose payload is the service_id followed by all elementary stream pids. Quite often this ecm is *identical* across many feeds. There lies the problem - the ecm is not sufficiently unique per channel/feed.

So I extended the fake ecm, adding a uint32_t "fingerprint" of the channel after the list of pids. The fingerprint is a crc32 over an extended list of pids, and more importantly, the enigma2 namespace for the channel. The namespace is a 32-bit value formed from the orbital position, frequency & polarisation. The namespace is also appended to the fake ecm. Those mods are in module-dvbapi.c

The BISS emu in module-emulator-osemu.c is updated to handle this extended fake ecm, whilst maintaining compatiblity with the previous scheme. SoftCam.Key is used, replacing the svid|pid key index with the crc32 channel hash.

Usage:
Remove old entries in SoftCam.Key.
On zapping to a new feed, in the live log you will see the emu searching for a key as before svid|pid. If no match is found, you'll see a suggested SoftCam.Key line in the log:
F DE1F1603 00 0000000000000000 7.0°E 12564H 20170923@1928

You'll notice it's annotated with the sat, frequency & time. Copy/paste that into your SoftCam.Key & insert the correct key. I store them in sorted sat/freq order, adding the feed name & event details by hand. It grows to become a nice log of your feedhunting activity:

Code:

#############################################
#                 BISS KEYS                 #
#############################################
;On every channel change, the emu makes a BISS key search, in this order, until a match is found:
;1st:  F SvIdEcmP 00 0102030405060708 ;(ServiceID<<16 | ECMPID) whenever ecmpid!=0x1FFF
;2nd+: F SvIdXPid 00 0102030405060708 ;(ServiceID<<16 | XPID) where X = any valid A/V pid for the service, searched in turn, in the order they appear in the PMT.
;then: F ChnlHash 00 0102030405060708 ;(channel hash) a (hopefully!) unique value which can be seen in OSCAM's live log whilst the emu searches for a match.
;last: F SvId1FFF 00 0102030405060708 ;(ServiceID<<16 | 0x1FFF) last chance saloon, effectively just search on the serviceID.
;If the key changes, either replace the key or place another line *below* the original entry
;Keys are re-read on stop-starting the emulator reader, or re-starting oscam.

F 00011FFF 00 0000000000000000 Default key (RAI) will be used for serviceID 0001 when no matching hash is found

F AE43A6F6 00 0000000000000000  12.5°W 11485H 20170923@1525 Derby-Birm City (EFL)
F 896A3EFA 00 0000000000000000  12.5°W 12678H 20170923@1532 F503 Cod2 (Montpellier - PSG)

F 9F5296B0 00 0000000000000000   0.8°W 11486V 20170923@1724 Connacht v Cardiff Vis 1 (Rugby)
F CAAE4226 00 0000000000000000   0.8°W 11500V 20170923@1726 Connacht v Cardiff World Feed (Rugby)
F E9ECF7B6 00 0000000000000000   0.8°W 11513V 20170923@1716 Tower OB TX4

F B1406A6A 00 0000000000000000   3.0°E     0H 20170923@1859 F32 Cod5 L1 (French Football)
F D06B4F53 00 0000000000000000   3.0°E     0H 20170923@1906
F 9E62C8B1 00 0000000000000000   3.0°E 10962H 20170923@2142 ITA 692 (Serie A Football)
F B242A483 00 0000000000000000   3.0°E 10985H 20170923@1918 BeIN - LCFC Vs LFC (Premiership Football)
F E8533582 00 0000000000000000   3.0°E 11524V 20170923@1903 E381 Via1 (French Football)

F 9679484D 00 0000000000000000   7.0°E 11104V 20170924@0859 D 408 # 2 Berlin (Marathon)
F E1EE1438 00 0000000000000000   7.0°E 11500V 20170923@0806 MOTO GP ARAGON
F DE1F1603 00 0000000000000000   7.0°E 12564H 20170923@1928 AUT10/ITTF-Austria-1
F B2A42DDC 00 0000000000000000   7.0°E 12569H 20170923@1129 Arqiva HD3

F F471D2EB 00 0000000000000000  10.0°E 10887V 20170923@1251 Zambia OB6 Encoder 1
F 8C0768BC 00 0000000000000000  10.0°E 11059V 20170923@1224 Rai ITA 475A
F A26143E5 00 0000000000000000  10.0°E 11043V 20170923@1241 Rai ITA77A
F 980634F3 00 0000000000000000  10.0°E 11055V 20170923@1354 RAI ITA 94 A
F FC6D9C33 00 0000000000000000  10.0°E 11062V 20170923@1225 Rai ITA 475B
F F1C53A0C 00 0000000000000000  10.0°E 11067V 20170923@1227 Rai ITA76B
F 84919CDE 00 0000000000000000  10.0°E 11107V 20170923@1356 RAI ITA88A
F 95D7FA42 00 0000000000000000  10.0°E 11163H 20170923@1154 GCUK Enc 1 (Scottish Football)
F AF95A273 00 0000000000000000  10.0°E 11500H 20170923@1247 GCUK Enc 5
F 96E924AE 00 0000000000000000  10.0°E 11500V 20170923@1407 MOTO GP ARAGON
F 96E924AE 00 0000000000000000  10.0°E 11500V 20170924@0750 MOTO GP ARAGON
F EABD1656 00 0000000000000000  10.0°E 12739V 20170923@1301 E-399 Via1

F B61C7EB2 00 0000000000000000  16.0°E 10997H 20170924@0735 EUTELSAT 16A S3-02
F FA02EC8D 00 0000000000000000  16.0°E 11000H 20170924@0736 ITA466 RM-1

F F47D3E5F 00 0000000000000000  21.6°E     0H 20170923@1802 MBC Feed

F E099AD52 00 0000000000000000  28.2°E 12683H 20170923@1427 SiS80 Coder 2 (Premiership Football)

Note:
I occasionally see incorrect frequency/polarity 0H. On those channels I see the namespace reported in enigma2's webif with lower 16 bits all 0's too, so suspect a bug in openatv/enigma2.

Patch placed here as I couldn't see how to insert an attachment (maybe I don't have rights, or am blind this morning lol). First patch oscam svn with oscam-emu.patch, then this one.:

Code:

Common subdirectories: oscam-svn-emu/cscrypt and oscam-svn-dev/cscrypt
Common subdirectories: oscam-svn-emu/csctapi and oscam-svn-dev/csctapi
Common subdirectories: oscam-svn-emu/devtools and oscam-svn-dev/devtools
Common subdirectories: oscam-svn-emu/Distribution and oscam-svn-dev/Distribution
Common subdirectories: oscam-svn-emu/extapi and oscam-svn-dev/extapi
Common subdirectories: oscam-svn-emu/ffdecsa and oscam-svn-dev/ffdecsa
Common subdirectories: oscam-svn-emu/minilzo and oscam-svn-dev/minilzo
diff -u oscam-svn-emu/module-dvbapi.c oscam-svn-dev/module-dvbapi.c
--- oscam-svn-emu/module-dvbapi.c	2017-09-24 09:30:50.415061296 +0100
+++ oscam-svn-dev/module-dvbapi.c	2017-09-24 09:25:00.000000000 +0100
@@ -2141,6 +2141,25 @@
 				er->ecmlen += 2;
 			}
 
+			//add more pids/IDs to better "fingerprint" the feed
+			i2b_buf(2, er->caid, er->ecm + 3 + er->ecm[2]);
+			i2b_buf(2, er->pid, er->ecm + 5 + er->ecm[2]);
+			i2b_buf(2, er->prid, er->ecm + 7 + er->ecm[2]);
+			i2b_buf(2, er->pmtpid, er->ecm + 9 + er->ecm[2]);
+			i2b_buf(2, er->onid, er->ecm + 11 + er->ecm[2]);
+			i2b_buf(2, er->tsid, er->ecm + 13 + er->ecm[2]);
+			uint32_t ensp=demux[demux_id].enigma_namespace;
+			i2b_buf(4, ensp, er->ecm + 15 + er->ecm[2]);
+
+			//generate a hash from the pid list
+			int32_t pidsCRC32=crc32(0x2600, er->ecm + 3, er->ecm[2] + 16);
+
+			pidsCRC32|=0x80000000; //set bit31: flag to BISS emu this is our CRC32, not a pid ie use full 32bit hash instead of (tsid<<16 | pid) as key index
+			i2b_buf(4, pidsCRC32, er->ecm + 3 + er->ecm[2]); //place crc32 after last stream pid
+			i2b_buf(4, ensp, er->ecm + 3 + er->ecm[2] + 4); //place namespace after crc32 hash (we'll use this to annotate the softcam entry with pos & freq)
+			er->ecm[2] += 8;
+			er->ecmlen += 8;			
+
 			cs_log("Demuxer %d trying to descramble PID %d CAID %04X PROVID %06X ECMPID %04X ANY CHID PMTPID %04X VPID %04X", demux_id, pid,
 				   demux[demux_id].ECMpids[pid].CAID, demux[demux_id].ECMpids[pid].PROVID, demux[demux_id].ECMpids[pid].ECM_PID,
 				   demux[demux_id].pmtpid, demux[demux_id].ECMpids[pid].VPID);
diff -u oscam-svn-emu/module-emulator-osemu.c oscam-svn-dev/module-emulator-osemu.c
--- oscam-svn-emu/module-emulator-osemu.c	2017-09-24 09:30:50.485058327 +0100
+++ oscam-svn-dev/module-emulator-osemu.c	2017-09-24 09:15:27.000000000 +0100
@@ -2420,15 +2420,48 @@
 }
 
 // BISS Emu
+void time2str(char * buf, uint8_t len)
+{
+	time_t rawtime;
+	struct tm timeinfo;
+	time(&rawtime);
+	localtime_r(&rawtime, &timeinfo);
+	strftime(buf, len, "%Y%m%d@%H%M", &timeinfo);
+}
+
+void namespace2str(char *buf, uint8_t len, uint32_t hash)
+{
+	//extract pos & freq from namespace hash. See
+	//https://github.com/openatv/enigma2/blob/master/lib/dvb/frontend.cpp#L496
+	//hash = (sat.orbital_position << 16);
+	//hash |= ((sat.frequency/1000)&0xFFFF)|((sat.polarisation&1) << 15);
+
+	uint16_t frequency, degrees;
+	char compass, polarisation;
+
+	degrees = hash **** 16;
+	if (degrees > 1800)
+	{
+		degrees = 3600 - degrees;
+		compass = 'W';
+	}
+	else compass = 'E';
+
+	frequency = hash & 0x7FFF;
+	polarisation = hash&0x8000 ? 'V':'H';
+
+	snprintf(buf, len, "%5.1f\xB0%c %5d%c", degrees/10.0, compass, frequency, polarisation);	
+}
+
 static int8_t BissECM(uint16_t UNUSED(caid), const uint8_t *ecm, int16_t ecmDataLen,
 					  uint8_t *dw, uint16_t srvid, uint16_t ecmpid)
 {
 	uint8_t haveKey1 = 0, haveKey2 = 0;
 	uint16_t ecmLen = 0, pid = 0;
-	uint32_t i;
+	uint32_t i, pidsCRC32;
 	
 	//try using ecmpid if it seems to be valid
-	if(ecmpid != 0) {
+	if(ecmpid != 0x1FFF) {
 		haveKey1 = FindKey('F', (srvid<<16)|ecmpid, 0, "00", dw, 8, 1, 0, 0, NULL);
 		haveKey2 = FindKey('F', (srvid<<16)|ecmpid, 0, "01", &dw[8], 8, 1, 0, 0, NULL);
 		
@@ -2437,13 +2470,29 @@
 		else if(!haveKey1 && haveKey2) {memcpy(dw, &dw[8], 8); return 0;}
 	}
 	
-	//try to get the pid from oscam's fake ecm ([sid] ([pid1] [pid2] ... [pidx])
+	//try to get the pid from oscam's fake ecm ([sid] ([pid1] [pid2] ... [pidx] [CRC32] [nameSpace])
 	if(ecmDataLen >= 3) {
 		ecmLen = GetEcmLen(ecm);
 		
 		if(ecmLen > 7 && ecmLen <= ecmDataLen) {
 			for(i=5; i+1<ecmLen; i+=2) {
 				pid = b2i(2, ecm+i);
+				if(pid&0x8000 && ecmLen==i+8) {
+					char tbuf[32], tbuf2[32];
+					pidsCRC32=b2i(4, ecm + i);
+					haveKey1 = FindKey('F', pidsCRC32, 0, "00", dw, 8, 1, 0, 0, NULL);
+					namespace2str(tbuf2, sizeof(tbuf2), b2i(4, ecm + i+4));
+					if(haveKey1)
+					{
+						memcpy(&dw[8], dw, 8);
+						cs_log("Key found F %08X ........ %s", pidsCRC32, tbuf2);
+						return 0;
+					}
+					//no key found matching our hash: put example SoftCam.Key BISS line for this feed in the live log
+					time2str(tbuf, sizeof(tbuf));
+					cs_log("F %08X 00 0000000000000000 %s %s", pidsCRC32, tbuf2, tbuf);
+					break;
+				}
 				haveKey1 = FindKey('F', (srvid<<16)|pid, 0, "00", dw, 8, 1, 0, 0, NULL);
 				haveKey2 = FindKey('F', (srvid<<16)|pid, 0, "01", &dw[8], 8, 1, 0, 0, NULL);
 				
Common subdirectories: oscam-svn-emu/.svn and oscam-svn-dev/.svn
Common subdirectories: oscam-svn-emu/tommyDS_hashlin and oscam-svn-dev/tommyDS_hashlin
Common subdirectories: oscam-svn-emu/toolchains and oscam-svn-dev/toolchains
Common subdirectories: oscam-svn-emu/utils and oscam-svn-dev/utils
Common subdirectories: oscam-svn-emu/webif and oscam-svn-dev/webif

I'll attach an arm build as used on my SF4008 as soon as I can find the attach button :thum:

Wilb · Sep 24, 2017

Awesome, I've recently started exploring feeds and had exactly the same frustration. Handily I also have a SF4008 too.

Sent from my MI 5 using Tapatalk

m35istan · Sep 24, 2017

Hi,
Thanks for your good work.
Could you explain how can we patch oscam?
I have enigma2 ( Vuplus Solo2 ). I can patch oscam-emu but i do not know how i will patch second time.
Thanks.

campag5242 · Sep 24, 2017

Hi m35istan - you should copy the patch text above into a new file, for example "newbiss.patch". Download oscam svn, apply the latest oscam-emu.patch, then apply my patch in the exact same way.

For the SF4008, I compile natively on a Raspberry Pi.

campag5242 · Sep 24, 2017

Just tried applying the patch myself. You should cd to oscam-svn, then:
patch -p1 < path/to/newbiss.patch

p1 is needed to strip my directory names from the paths included.

m35istan · Sep 24, 2017

campag5242 said:
Hi m35istan - you should copy the patch text above into a new file, for example "newbiss.patch". Download oscam svn, apply the latest oscam-emu.patch, then apply my patch in the exact same way.

Hi,
I am using simplebuild for patch. So i do not patch second time.
Your patch ready for with "biss.patch" name.
I try this command but not apply.

Code:

patch -p0 < /root/oscam-svn/biss.patch

kebien · Sep 24, 2017

But is not the same idea around constant CW?
I hardly use Biss entries in softcam.key file.
You can do a constant.cw entry that uses video pid,PMT pid,sid as variables and generally works very well.
Just as OScam requires :
#CAID

rovider ID:Service ID

MT ID:ECM PID:Video PID:key (16 Bytes seperated by spaces)

But in my view,and as a strong suggestion to avoid this problem since this problem arises,I build the key selection using the channel name alone,as in :
CHANNELNAME KEY ;
And entering the new key through screen menu
It proved to be foolproof for 15 years.

campag5242 · Sep 24, 2017

@ m35istan see my posting a few minutes earlier than yours - use p1.

@ kebien. I was using constant.cw before creating this patch. My keyfile was filled with lots of entries like:
2600:000000:0001:0020:1FFF:0134:... repeated
2600:000000:0001:0020:1FFF:0200:... ditto
& many other combinations popular amongst the uplink crews. Only GCUK seemed to play nicely...

Anyways, the above is working well for me - since yesterday, a bit shy of your 15 years lol. I hope it's of use to others

kebien · Sep 24, 2017

I didn't mention the 15 years to brag.
I meant from the beginning I tried to make developers think differently about the solution.
So I resorted to make a separate plugin for biss using the channel name with editable name and key from remote,from the days of enigma 1.And added other parameters that had maybe used only once.But other option is just to rename then channel to your liking,then enter the key for it.
Since Biss needs no algorithm,is not really an emulator,it just sets a key to deliver to the CA device using I/O cmds.

campag5242 · Sep 24, 2017

How do you get at the current channel name inside oscam? I only stumbled across this namespace variable thus far. I'd love to automatically annotate the key line template with the provider, channel name & even signal strength too.

kebien · Sep 24, 2017

is not OScam,is a separate plugin only for biss,can coesxist with oscam given you do not set OScam for biss,remove the 2600 entries
Believe me,is harder to make this work with OScam,a separate plugin makes your life easier

Anyway,was not possible to do this 15 years ago with developers at the time (no oscam back then,only the closed sources emus,like evocam and others)
No point on resurfacing this stuff now.
Any improvement is a good improvement,kudos for the work
I am only using constant.cw now,since I do not have that problem since I am not hunting feeds a lot this days

budiarno · Sep 24, 2017

This is nice, I think I'm interested in implementing your patch for the Transponder part (Frequency

olarisation:SymbolRate).

campag5242 · Sep 24, 2017

I wish the transponder detail included the SR (as well as provider, channel name, rssi/dB). Is it possible?

kebien · Sep 24, 2017

campag5242
can't you just poll the frontend for that information?
Beside,isn't that information in lamedb?

campag5242 · Sep 24, 2017

Hi kebien - I expect so (I'm not experienced here), but try to keep things simple & use only what comes via the dvb-api. Accessing lamedb will be an issue for portability (can think of SamyGO for one).

Now you make me think... I don't know whether on eg SamyGO there's similar TP info encoded in a namespace variable... must check.

m35istan · Sep 24, 2017

Hi,
I try patch but result is here :

I patched first emu and make oscam. It is OK.
After try your patch. Give some error.
After make oscam also give some error.

campag5242 · Sep 24, 2017

Hi m35istan. I guess windows has added some unwanted CRLF line termination to the patch. I've uploaded the patch & the two source files already patched here for you: http://dropmefiles.com/eA91y

m35istan · Sep 24, 2017

campag5242 said:
Hi m35istan. I guess windows has added some unwanted CRLF line termination to the patch. I've uploaded the patch & the two source files already patched here for you: http://dropmefiles.com/eA91y

Hi,
Thanks for interesting. But if it possible give me oscam bin for enigma2.
Because when i patch'ed ubuntu give me x86_64 oscam bin.
I am use simplebuild for cross compile. But i do not use your patch with simplebuild.
Thanks.

campag5242 · Sep 24, 2017

Hi m35istan - taking the two source files from the zip should be all you need.

If you've successfully built an oscam for your e2 box before, then there should be no problem.

I've only built for arm box Octagon SF4008. I cannot help with your cross-compilation issues.

zayden · Sep 25, 2017

kebien said:
I didn't mention the 15 years to brag.
I meant from the beginning I tried to make developers think differently about the solution.
So I resorted to make a separate plugin for biss using the channel name with editable name and key from remote,from the days of enigma 1.And added other parameters that had maybe used only once.But other option is just to rename then channel to your liking,then enter the key for it.
Since Biss needs no algorithm,is not really an emulator,it just sets a key to deliver to the CA device using I/O cmds.

Is this plugin public somewhere ?

Sent from my SM-N910W8 using Tapatalk

oscam - a better BISS?

campag5242

Wilb

m35istan

Registered

campag5242

campag5242

m35istan

Registered

kebien

Registered

campag5242

kebien

Registered

campag5242

kebien

Registered

budiarno

Registered

campag5242

kebien

Registered

campag5242

m35istan

Registered

campag5242

m35istan

Registered

campag5242

zayden

Registered