Here is some very relevant info for all cache-EX users & oscam users in general
There have been big changes in cache-EX in recent days..
Originally posted by capncook & blueven @ streamboard
QuoteDisplay MoreThis is overall #8895 changeset:
FIX:
- Code optimization/cleaning and improvement in ecm handling.
- Make ecm answers thread-safe.
- Make accessing ecm request by cacheex code thread safe. This help increasing hits!
- thread safe fixs above, also fix many possible segfault scenario's.
- Fix group matching when checking in cache. Now groups works as expected.
- Set oldest reader in LB mode only when request is effectively send and when reader is not used as fallback.
- Fix a possible seg fault in camd35 and gbox protocols.
- Fix incorrect ecm handling in cccam code, causing seg faults.
- Not use provid = 000000 if it is not specified in "ident" parameter (reader setting).
- Fix dyn wait_time when pushed cacheex ecm use different prid and/or srvid than requesting ecm (but same caid+hash).
- Fix cache stats counters.
- Print msg "wait_time over" only when answer from cacheex.
- In chk_dcw, push for cacheex only E_FOUND and not E_CHACHEX, because cacheex cascade push was already done.
- Set ecm time to 1ms if real ecm time <1ms.
NEW:
- New parameter lb_whitelist_services: now allow and works with no srvid in service definition. Use it for whitelist reader in LB when it fails. (No need specify srvid in service definition, caid is enough!)
- New parameter fallback_percaid parameter in reader section.
fallback_percaid= <CAID>[:<ident>[,ident]]...[;<CAID>[:<ident>[,ident]]...].... wildcard CAIDs with two-digit CAIDs possible, default none.
fallback_percaid overrule fallback parameter.
- New parameter fallbacktimeout_percaid in global section.
fallbacktimeout_percaid=CAID1:time1[,Caid2:time2]... wildcard CAIDs with two-digit CAIDs possible, default none.
- LB: takes into consideration "fallback" and "fallback_percaid" parameters. (Details here)
- LB: NOT block reader if "not_found" due to ratelimit check.
- Preciser ecm-time computation for LB stats!
- Reworked ecm pending method. Now "pending handling" is based on ecm answer and not more on ecm request. This avoid timeouts due to first client failure or disconnect!
- Immediately call to fallback readers when all "not fallback" readers already answered. So, we don't have to wait fallbacktimeout if there aren't other answers to wait. This improve ecm time when using fallback!
- A reader got timeout imemdiately after clienttimeout if not answer. Not wait anymore other ecm for adding previous rc to stats.
- Rework on cacheex mode-1. Now re-request possible! More, cacheex-1 answers are now added to cache stats! (Details here)
- Removed max_count parameter from cache setting. (Details here)
- Print in log "cache3 (... ms) by ..." even when cache3 from cacheex mode-3 client or csp
- Answers from cacheex-1 readers are shown in log as "cache3" (before was "found")
- Added more debug lines (-d256) to see as oscam internally works!
I am using #8902 at the minute and the improvements are massive..
One warning, cpu load will increase after updating to post #8895 SVN..
Here is an explination by capncook @ streamboard
QuoteDisplay MoreCongrats blueven for hitting this to trunk! Bier zustossen
I think the whole "high CPU" discussion above is there, because people 'just upgrade oscam', and have no clue about the big amount of changes coming with this release.
--
To all having a high(er) CPU load: max_count has been removed, so cache-size can only be influenced by max_time. Therefore, if you had cache limited by max_count, this limit is released.
In general: cache can only be cleaned from memory if its older as clienttimeout + 3sec. So if you have clienttimeout set to 5000ms, lowest max_time that can be used = 8sec !
If with that lowest setting, your size is still not at the level you want, lower cacheex_maxhop or filter caids. (- or get rid of a bunch of peers, remember that just 1 peer can fill upto 20.000).
Since oscam is using linked-list methods to store its cache, cpu goes up quickly.
I tested several sizes on a i7-3930K (6x3.2Ghz), and cpu goes bezerk when i hit around 20.000 size. In this system, oscam seems most comfortable with a size of 10.000 (still 50%CPU), which gains the most hits from cache (locally).
When adding more, hits% goes down, while CPU usage grows the more cache is added. (When you use oscam as a forwarding server only, without local hits, this can be higher, since the cache-forwarding is less affected by size.)
--
The culprit still is: if we want larger caches with less load, someone has to change linked-list(s) to (eg) hashed tables, which isnt an easy job as i've been told
I hope that all oscam users take the time to leave some positive comments for blueven for the hard work & time that has gone into these improvements
