balance the load on leader preferred replica - first replica of each partition preferred replica is evenly distributed among brokers tries to make the preferred replica as the leader and hence balancing the load

followers that lag behind replica lag time max ms is removed from the ISR

follower failure is handled by removing the lagging or failed followers from the ISR and considering only the rest and moves the high watermark

a log is committed when all replicas are in sync - ISR - then we move watermark

only data upto high watermark is visible to the consumers

leaders&followers leader commits log with an epoch(indicates generation of lifetime of a log) follower fetch with offset no to sync with leader leader reponds with new record logs with epoch in the next fetch request, leader moves high watermark based on offset in request (n-1) in that response it passes the high watermark to the follower

how data is received client -> socket -> n/w thread(lightweight) -> req queue -> i/o thread(crc check & append to commit log[.index and .log file]) -> resp queue -> n/w-> socket

purgatory map -> used for req until data is replicated

how data is fetched client -> socket -> n/w -> req queue -> i/o thread (fetch ranges are calcualted with .index) -> res queue -> n/w -> socket

purgatory map -> waits until is arrived based on consumer config

also follows zero copy - meaning data fetched from file directly to n/w thread - mostly page cached

if not cached -> may need to use tiered storage

consumer properties => same as producer, time and batch size.

producer key properties => linger time -> linger.ms batch size -> batch.size

these determine the throughput and latency of kafkaproducers

independent work, without depending others

Breakdown

1. Function or purpose
2. Interconnections or relationships - can change system massively by changing the information flow within it
3. Elements - low level way of changing like fire/replace, rarely make a difference otherwise they don't. <br /> 4.All of this produces behaviour of the system. <br /> 5.Event - one snippet of the behaviour of the systems, most news focus on this event, gives us no understanding of long term behaviour of the systems or why they are doing what they are doing.

deep insight -systems thinking - system behaviour comes out of the system, its interrelationships, its goals, not the elements of the people or actors in it

splitting up of tasks stops being a useful strategy if part of the tasks can't be further splittable

integration test with containerization looks like exactly what I need

inuse is calculated by allocated-free allocations

runtime scales the value, so reported allocations can be approximately equal to actual irrespective of the MemProfileRate

we should look at the percentage rather than individual functions since the optimizing the latter won't probably make any difference

again use Go1.18 atleast to avoid these errors

better to use Go1.18 versions to avoid these limitations

interesting so if data is small, it is managed by the cpu registers by the compiler, stack is not used

debugging should be done with compiler optimizations disabled

do we have any custom implementations of these profiles available publicly

very interesting

haven't heard about this, but looks great

i agree on this

would like to know more on why Go isn't suited for webservers

golang project structure, is this a more idiomatic approach to build microservice in go