Hypothesis: If the model’s input features look materially different for grade A/B deals that went UTP versus those that did not, it may suggest the model is not weighting those features appropriately. Conversely, if the feature distributions are similar, the model may lack the information needed to separate these cases.

Feature Comparison: Are Bad Predictions Distinguishable by Their Features?

Model Discrimination by Segment (ROC AUC)# How well does the model separate UTP from non-UTP customers in different segments? ROC AUC measures the model’s ability to rank-order risk correctly (0.5 = random, 1.0 = perfect). Some sub-sections also include PR AUC (Precision-Recall AUC), which focuses on the model’s performance specifically on the positive (UTP) class – this is particularly informative when the UTP rate is low, as it penalises false positives and false negatives more directly than ROC AUC. Segmentation sourced from int_customer_segmentation, joined as-of deal close date.

By UW Process Typ

By Approved Amount (Deal Size)

Industry group from int_customer_segmentation (e.g. Apparel & Fashion, Health & Beauty). This is one level above the more granular business sector. (function(root) { var docs_json = {"bf743a33-d651-471f-bac1-85b23c493483":{"version":"3.8.2","title":"Bokeh Application","config":{"type":"object","name":"DocumentConfig","id":"b12d50bf-88b0-4885-bc3b-a0030966c25f","attributes":{"notifications":{"type":"object","name":"Notifications","id":"6e91f9b1-2d52-44e7-bc22-fc59eaa78ce4"}}},"roots":[{"type":"object","name":"panel.models.plotly.PlotlyPlot","id":"db62437a-60ea-4359-99f3-e173fedf9cf6","attributes":{"name":"Plotly00170","subscribed_events":{"type":"set","entries":["plotly_event"]},"stylesheets":["\n:host(.pn-loading):before, .pn-loading:before {\n background-color: #c3c3c3;\n mask-size: auto calc(min(50%, 300px));\n -webkit-mask-size: auto calc(min(50%, 300px));\n}",{"type":"object","name":"ImportedStyleSheet","id":"e9f47259-3dd2-48e1-92fb-b79b66998df6","attributes":{"url":"https://cdn.holoviz.org/panel/1.8.7/dist/css/loading.css"}},{"type":"object","name":"ImportedStyleSheet","id":"c353abbd-41df-4fe6-aa99-005013c83f4b","attributes":{"url":"https://cdn.holoviz.org/panel/1.8.7/dist/bundled/plotlyplot/maplibre-gl@4.4.1/dist/maplibre-gl.css"}},{"type":"object","name":"ImportedStyleSheet","id":"714166e3-cc59-4d20-86e7-c5bf33875d0d","attributes":{"url":"https://cdn.holoviz.org/panel/1.8.7/dist/bundled/theme/default.css"}},{"type":"object","name":"ImportedStyleSheet","id":"779c5e5b-a397-43b6-bbe9-d26dfa87e934","attributes":{"url":"https://cdn.holoviz.org/panel/1.8.7/dist/bundled/theme/native.css"}},":host { --plotly-icon-color: gray; --plotly-active-icon-color: #2a3f5f; }"],"margin":[5,10],"align":"start","data":[{"type":"map","entries":[["hovertemplate","Industry Group=%{x}<br>Bad Prediction Rate (%)=%{text}<extra></extra>"],["legendgroup",""],["marker",{"type":"map","entries":[["color",["#736e6e","#736e6e","#736e6e","#736e6e","#59c6df","#59c6df","#59c6df","#ff8d98","#736e6e","#736e6e","#736e6e","#736e6e","#736e6e","#736e6e","#736e6e"]],["pattern",{"type":"map","entries":[["shape",""]]}]]}],["name",""],["orientation","v"],["showlegend",false],["textposition","outside"],["xaxis","x"],["yaxis","y"],["type","bar"],["error_y",{"type":"map","entries":[["array",[31.25,24.550000000000004,28.650000000000002,28.650000000000002,10.020000000000001,3.9399999999999995,5.12,3.09,56.15,56.15,48.99,39.03,79.35,32.44,29.91]],["arrayminus",[21.269999999999996,9.59,7.47,7.47,4.879999999999999,2.42,1.8399999999999999,0.8699999999999999,0.0,0.0,0.0,0.0,0.0,0.0,0.0]],["symmetric",false],["thickness",1.5],["type","data"],["width",5]]}],["texttemplate","%{text:.1f}%"]]}],"layout":{"type":"map","entries":[["template",{"type":"map","entries":[["data",{"type":"map","entries":[["histogram2dcontour",[{"type":"map","entries":[["type","histogram2dcontour"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}],["colorscale",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]]]}]],["choropleth",[{"type":"map","entries":[["type","choropleth"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]],["histogram2d",[{"type":"map","entries":[["type","histogram2d"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}],["colorscale",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]]]}]],["heatmap",[{"type":"map","entries":[["type","heatmap"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}],["colorscale",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]]]}]],["contourcarpet",[{"type":"map","entries":[["type","contourcarpet"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]],["contour",[{"type":"map","entries":[["type","contour"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}],["colorscale",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]]]}]],["surface",[{"type":"map","entries":[["type","surface"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}],["colorscale",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]]]}]],["mesh3d",[{"type":"map","entries":[["type","mesh3d"],["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]],["scatter",[{"type":"map","entries":[["fillpattern",{"type":"map","entries":[["fillmode","overlay"],["size",10],["solidity",0.2]]}],["type","scatter"]]}]],["parcoords",[{"type":"map","entries":[["type","parcoords"],["line",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scatterpolargl",[{"type":"map","entries":[["type","scatterpolargl"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["bar",[{"type":"map","entries":[["error_x",{"type":"map","entries":[["color","#2a3f5f"]]}],["error_y",{"type":"map","entries":[["color","#2a3f5f"]]}],["marker",{"type":"map","entries":[["line",{"type":"map","entries":[["color","#E5ECF6"],["width",0.5]]}],["pattern",{"type":"map","entries":[["fillmode","overlay"],["size",10],["solidity",0.2]]}]]}],["type","bar"]]}]],["scattergeo",[{"type":"map","entries":[["type","scattergeo"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scatterpolar",[{"type":"map","entries":[["type","scatterpolar"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["histogram",[{"type":"map","entries":[["marker",{"type":"map","entries":[["pattern",{"type":"map","entries":[["fillmode","overlay"],["size",10],["solidity",0.2]]}]]}],["type","histogram"]]}]],["scattergl",[{"type":"map","entries":[["type","scattergl"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scatter3d",[{"type":"map","entries":[["type","scatter3d"],["line",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scattermap",[{"type":"map","entries":[["type","scattermap"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scattermapbox",[{"type":"map","entries":[["type","scattermapbox"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scatterternary",[{"type":"map","entries":[["type","scatterternary"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["scattercarpet",[{"type":"map","entries":[["type","scattercarpet"],["marker",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}]]}]],["carpet",[{"type":"map","entries":[["aaxis",{"type":"map","entries":[["endlinecolor","#2a3f5f"],["gridcolor","white"],["linecolor","white"],["minorgridcolor","white"],["startlinecolor","#2a3f5f"]]}],["baxis",{"type":"map","entries":[["endlinecolor","#2a3f5f"],["gridcolor","white"],["linecolor","white"],["minorgridcolor","white"],["startlinecolor","#2a3f5f"]]}],["type","carpet"]]}]],["table",[{"type":"map","entries":[["cells",{"type":"map","entries":[["fill",{"type":"map","entries":[["color","#EBF0F8"]]}],["line",{"type":"map","entries":[["color","white"]]}]]}],["header",{"type":"map","entries":[["fill",{"type":"map","entries":[["color","#C8D4E3"]]}],["line",{"type":"map","entries":[["color","white"]]}]]}],["type","table"]]}]],["barpolar",[{"type":"map","entries":[["marker",{"type":"map","entries":[["line",{"type":"map","entries":[["color","#E5ECF6"],["width",0.5]]}],["pattern",{"type":"map","entries":[["fillmode","overlay"],["size",10],["solidity",0.2]]}]]}],["type","barpolar"]]}]],["pie",[{"type":"map","entries":[["automargin",true],["type","pie"]]}]]]}],["layout",{"type":"map","entries":[["autotypenumbers","strict"],["colorway",["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"]],["font",{"type":"map","entries":[["color","#2a3f5f"]]}],["hovermode","closest"],["hoverlabel",{"type":"map","entries":[["align","left"]]}],["paper_bgcolor","white"],["plot_bgcolor","#E5ECF6"],["polar",{"type":"map","entries":[["bgcolor","#E5ECF6"],["angularaxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""]]}],["radialaxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""]]}]]}],["ternary",{"type":"map","entries":[["bgcolor","#E5ECF6"],["aaxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""]]}],["baxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""]]}],["caxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""]]}]]}],["coloraxis",{"type":"map","entries":[["colorbar",{"type":"map","entries":[["outlinewidth",0],["ticks",""]]}]]}],["colorscale",{"type":"map","entries":[["sequential",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]],["sequentialminus",[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]],["diverging",[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]]]]}],["xaxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""],["title",{"type":"map","entries":[["standoff",15]]}],["zerolinecolor","white"],["automargin",true],["zerolinewidth",2]]}],["yaxis",{"type":"map","entries":[["gridcolor","white"],["linecolor","white"],["ticks",""],["title",{"type":"map","entries":[["standoff",15]]}],["zerolinecolor","white"],["automargin",true],["zerolinewidth",2]]}],["scene",{"type":"map","entries":[["xaxis",{"type":"map","entries":[["backgroundcolor","#E5ECF6"],["gridcolor","white"],["linecolor","white"],["showbackground",true],["ticks",""],["zerolinecolor","white"],["gridwidth",2]]}],["yaxis",{"type":"map","entries":[["backgroundcolor","#E5ECF6"],["gridcolor","white"],["linecolor","white"],["showbackground",true],["ticks",""],["zerolinecolor","white"],["gridwidth",2]]}],["zaxis",{"type":"map","entries":[["backgroundcolor","#E5ECF6"],["gridcolor","white"],["linecolor","white"],["showbackground",true],["ticks",""],["zerolinecolor","white"],["gridwidth",2]]}]]}],["shapedefaults",{"type":"map","entries":[["line",{"type":"map","entries":[["color","#2a3f5f"]]}]]}],["annotationdefaults",{"type":"map","entries":[["arrowcolor","#2a3f5f"],["arrowhead",0],["arrowwidth",1]]}],["geo",{"type":"map","entries":[["bgcolor","white"],["landcolor","#E5ECF6"],["subunitcolor","white"],["showland",true],["showlakes",true],["lakecolor","white"]]}],["title",{"type":"map","entries":[["x",0.05]]}],["mapbox",{"type":"map","entries":[["style","light"]]}]]}]]}],["xaxis",{"type":"map","entries":[["anchor","y"],["domain",[0.0,1.0]],["title",{"type":"map","entries":[["text","Industry Group"]]}],["gridcolor","#145640"],["griddash","dot"],["showgrid",false],["zerolinecolor","#145640"],["tickangle",-45]]}],["yaxis",{"type":"map","entries":[["anchor","x"],["domain",[0.0,1.0]],["title",{"type":"map","entries":[["text","UTP Rate among Grade A/B (%)"]]}],["gridcolor","#145640"],["griddash","dot"],["zerolinecolor","#145640"]]}],["legend",{"type":"map","entries":[["tracegroupgap",0],["entrywidth",150],["itemwidth",40],["orientation","h"],["traceorder","normal"],["x",0.5],["xanchor","center"],["y",-0.3],["yanchor","bottom"]]}],["title",{"type":"map","entries":[["text","Bad Prediction Rate by Industry Group (Top 15)"],["font",{"type":"map","entries":[["size",15]]}],["x",0.5]]}],["barmode","relative"],["font",{"type":"map","entries":[["color","#1c1719"],["family","Montserrat, Trebuchet MS"]]}],["margin",{"type":"map","entries":[["b",200],["l",40],["r",30],["t",100]]}],["yaxis2",{"type":"map","entries":[["gridcolor","#145640"],["griddash","dot"],["zerolinecolor","#145640"]]}],["colorway",["#7e7ced","#6ddaa8","#f9782f","#d57bf5","#cdd03e","#59c6df","#ff8d98","#ffd90c","#5694f1","#778ad0","#76c299","#d2864e","#cd8bab","#96b049","#5fb5c0","#cd8b8b","#a48d1d","#64acdf","#b7b6d4","#b6d0b8","#eabfa1","#dfccd7","#c9cb74","#b6d4d0","#ebcccc","#e3cf92","#b2d0e1","#4b4a86","#145640","#8a5733","#86556d","#4b5825","#256374","#865555","#675811","#2e5185"]],["height",600],["images",[{"type":"map","entries":[["sizex",0.13],["sizey",0.13],["source","data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA+kAAAEYCAYAAAAgbLfqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAACxCSURBVHgB7d39tSS1tffxH3fx/zM3AosIwBEgIjCOYIoIgAimHQHjCLocARBByxEwjqCLCMARnKf3rdNrDofz0t1V0pZU389aWmeYGXuq1GpJW68SAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGzLJwL6dCc89tkpTQKA9b05pXhK4ZQ+v/8Z7v8syN+kuQ68Vjilo3yMp/SNAACb86kAbEESATqAdYVTeqs5OI8CAACrIEgHtuHfAoB1xFN6JwJzAACyIEgHtiEJAJaJIjgHACA7gnSgf5MI0gHczvabW3D+nQAAQHYE6UD/kgDgNuGUfjylLwQAAIogSAf6x350ALcIp3RQHaezAwCwGf8jAL1LAoDrBBGgAwDggiAd6FsSV68BuI7tQSdABwDACUE60DeWugO4lh0SFwQAAFwQpAN9SwKAywVxijsAAK4I0oF+TSJIB3CdvQAAgCuCdKBfSQBwuXBKUQAAwBVBOtAv9qMDuMY7AQAAdwTpQL+SAOByUQAAwB1BOtCnJK5eA3C5KE50BwCgCgTpQJ9Y6g7gGl8IAABUgSAd6FMSAFzuSwEAgCoQpAP9mUSQDuA6QQAAoAoE6UB/kgDgOix3BwCgEgTpQH/Yjw7gGkEAAKAaBOlAf5IA4HJBAACgGgTpQF+SuHoNAAAAaBZBOtAXlroDuNYbAQCAahCktyGe0l3n6TthDUkAcB2vIH08pU8Kpc8EAEAjCNJRi78JS00iSAcAAACaRpCOWkSx5HKpJAAAAABNI0hHTQZhCfajAwAAAI0jSEdNWPK+TBIAAACAphGkoyZRLHm/VRJXrwEAAADN+1RAXb7WfOLvUl+pnHhK7+SLpe4AAABABwjSUZu3WidITyrnC/lLAgAAANA8lrujNhbwtrbk/Uv5mkSQDgAAAHSBIB21sQA9qi1RvpIAAAAAdIEgHTVq6ZT3KP+Zf/ajAwAAAJ0gSEeN7PC4Vpa8R/lLAgAAANAFgnTUyAL0Gg5ju4T3fvQkrl4DAAAAukGQjlp9rfrVsH+epe4AAABARwjSUau3qh9XrwEAAABYFUE6atXCKe/es/2TCNIBAACArhCko2a1L3mvYT86AAAAgI4QpKNmNS95D/Jf7s5+dAAAAKAznwqo13nJe1J92I9+uzf6eIL/mwfpLw/+Tnjhf//7g/TfB7+e7n9+EAAAqF14kB73Bc7//ZTp/ue5HzCJPgA6Q5CO2tmS96T6/E2+kuq/eu0ciFv6/P5n0PON7pqm+2SN9X/uf9JwAwBQ3nnSJWjuD0S9HIQvNeljH+DfD36NPpz7l+E+nQd2gq7zlSrWQpC+U91+El/8nGzJ+3eqT5SvGpe6nytNG8CI8l1tEO5TfPB75xF2y7ukNlYi2CCV96qN95rzbkusLP8gPz9rblueE7VuHfS5fFjZ3imv6ZRG9a+GumKn+kW1cXvMpf6pOvugVodamfz8/mdQWUEf+wDnPqS1Y0lz/Wo/J7XF8tSzP7yTH6vbouazoM7BOSpwV3l6r/yipLsNp6i6BJEnZ9ZoDKd0OKXfVEd5uTTZ8+5V9wGFUf75VOMgWW6DfPM8vPJ8O+fnaykdtExQ3ud7Ke11uejwfI9TVP0G+efTmimoHucg8qA28u4XzYOxQW0I8s2v0qLmz+eoft6pO3eVp6Pyi5LuNpx2qssg3/w4yl/UXHm2Fpi/lKd71XHWwGMH+ebNQdtj5cErv/cXPN/O8flaSwctE5T3+ZaWhYe86+OD6jfIN488y0cOrQXmzyUL2AeV2Y53qyDfPCqhdHnCQncNpNxf6ijpbsPpqLr8KN/82MtPVPuN8WvpoLmxroU1WN55ErUdUfXn9c7x+VpLBy0TlPf5Xkp7XWdX8NmeS0F1G+SfRz3ktfV736mfgfpzOmr+3gXVJ8g3b3LyKk9V4wq2ddS8XLYHQXXNcEb58tiPHjV3dg/qP2CLmhvpo+oI1kf57wnfUh33Vn4mtXtrA/zVcH7EIJSQ5LOnOmhuHy2Y2qnumedbBM1luOZgvTcWnFt+79RfeVqEIH0dXwq51RIknK8N85RUTtB2gvPHguZG+hf5NtTW6f6nfFnguoXGM8g3yPiHgNtZXfEv+fpWdLRLKP05nw/TrGXwuoRB2+z7lBI19692os54EkH6OphJz6+WgZAoX0llRs/PS4+sQY7aNhuYsXx4Jz8lDqh8ifepsqVE+Zm0jVPIkdcoX+fDRJHPpLKf87kvsMVDRIPmQJ1Z9XXZYJ7la43nAFWDIH0d1igFIaeoOvLY+370Ekvdoz6ObuKjneaOSlB5NkP2k3x5LgMvxXMgJglY7oP8y5J3O9m7UituopjpPBs058UWByrWZOXIgnPviYcmEKSvh9n0/GrI4yhfSXmdRzeD8JQgvxkF7yXvQX2vqojyLfcsdcdavMtSFCuwckrKz5a2M9P5R+cl//SRbhM0D3RE4SIE6euJQm7eo/NRvibla5yt8bFT6xndvIw11KVnXZP8Z8g8Z5pz81wpMMrnECj0KWmeUfe0hZU3HkblrSuCmDF+TdQcqDM5d7kgBjeu9qmwlpx7pie9PDJu/3ZU/6LmYNLr9FrvCjkpjyAqz1vsNOfZNyrHZtOj/ET5fgdzCfLdR+t92Bf6Y2XKcxbU2svv1V9d4S3nKomoebCeQ7xeFzTn1U6sgnpNEH3Mbt01lKJ8vFdb+bQkeY7uHuT77oPWFzQv374j3Zz2Ksv7Xtqd+jPILz+Put6u0LP1kA5aJijv872U9rqdBVrUFX82yDdPPMvyS76VdEe6Kf2gMoKkO8d0C6uHjoWer9Q7FcNy93VF+fhS2+G15N0qmihfSesKYnRzDYPKBuree9N7vGLpnfwwC4Mcari68VthTblW3Fj9x1a329nkESsQnmZ9oyDchCB9XR7BctC2DvaI8qkIo3wlrbsPLYgAfU2Dyo2mW2fKcwmpff962osX5fc9mMS1a8inhqsbo7CGSXnqCgvQd8JS1iZan4pA/SMrW+zbX4AgfV1R5b+gUdszqLyerl47X4ERhDV9pzLbMSxA997D/Fb98JztSwLysboiyZfnKpWe5FhxQ4C+LpswKzVYX7sgytZiBOnrKz2r3VNn+VIeAbP3aoWk9bD8KB/r9JQoKz/JV1QfA4RBviP9LHVHbjVcx8bs4nJJ6yJAz2MQgbo5CIsRpK+vZIdvq0vJoso2+kG+Qfqk9Rpolh/lZeWyxP70JP8Zsh7K0SA/o7h2Dfkl+dcVnge+9mDUunUFAXpeVt63vIJkEBNBqyBIX1/JfelbDrZKvnuUr6R1BNEwl2ADOjvl5z1DZqt4Wp8h81yJxLVrKMW7rPV42GRJa9b1UfQDSthpm5NoQWxxWQ1B+vqsg16qMdrSqe6Plexce+fzWvvRWX5UjnVKg/JK8p2JtXqu5RmyQb4HxiUBZdj2GO/DJgfhFknr1fNB5a8M3TI78T1oW6xvHoRVEKTnUWqWd8sz6SUHQ6J8JS1nI5tBKMXKZonRZO8ZMu8DFZfwnEVnLzpKquE6tpbrCk9r1vFbDBo9ldr+VosgBuNWRZCeR4mZ16htLx8rdQ2UDQYE+UlaPooexPI2D4PyD/C8l+8MmX0/otoT5Pfck7h2DeV51xVRXMd2rUnr1RWlDjXFH0Vt50yGKAaBVkWQnkdUfls81f2xUoMhntZY6s7+ID+5v6c1zJC1WL48nzkJKM/qip/li0NLr7PWipsgBuo9WXuzhUk1+por+1TIIdynSflEwRr875V3dsB7P3rSMkHtLD/6/T5Nev678+Y+ldzusMSg/GV0lG/jGDV/Fp6zdNeK8sNSd3gZ5TvAb//2Tm3VFZ6S1tHaeTQP+wHPlZWgud05/6yZPZ9dy/aN2vZSOTp/FtiYu0ZTzuUtX0i6I/1fisrrN0l3Tumo5fbK+4xL0kHzEkwbbAm63vkKwt39/1et77lTft7vv1M7Bvnl017r2EnZn7WXdNAyQXmfr0R5eeggFXv+p9JOvgb5vn/pz36Qsj/rkmR9rIPmPnPUbQH3uS/wXnX3BaKWCcr7fFtMVftE9as+E59hy8pyLe2yyuwHwYzKNzoZ5TsCPWrZuwWtE+ivadJ8EE6O/ZFB82f2TnWN6Np7/q/yivItqyXecS2WT1E+vtI6s2NR676DHerltV/VtmvknFmdtGxfb5BfPTpq/fYtyreumE7pM/kZ1MZhXpZHk5YJmj/roPr8pPm7/0Hb6QskzW3ArYLq69O1roU4uGreoyy3pt+Uz0HSnXN6X8Ez5M7nnaQ7xzRomb3yPt816aiygdFw/2/W8v5R+R0k3TmmFvabBvnlz1H1GuWXL0F1C/LLm73y8FwhZinKzyDfd78kHbSOd8r/rNckK3c7lV2eHtVPXyBkfK6tJizU0of9OEWtL0i6c06H+2c5VPAsufK5hvcLWuaovM93adrJR5Bv8PEwvVd+30m6c0wH1W8vv/wZVK9RfvkSVLcgv7zZK4+dpDvHdJCfQb7vfkkatFxQ3me8Nlkb6Ll3fFAdfaKDbhcyPtdWExZq6cN+nHLsSx8k3Tmn4f5ZvIOCc8oRAL2Rij3/U+mgZQblfb5L0lF1HHBYQzk9Kj8rs1ueIbuEfQ69fv5LjPIrM0F1C/LLm73yqKGu8ArYBvm+92vpqHXslf9ZL32fqDoEzXfFe+dJ1G1C5ufaYqoaV7Dl9Tet7638pfufo+o4pTVHnnjtzzxbevVajrJ3jUnr7b9dygZxvE9VDcrfKa3hOraal7wP8gsIk4B6WF3xL/nKebhuy9a4/SGojpU7tu/8r6qn/ptO6e/yv2Gjhn48sIraR2FeSjZSvWbH3Ht219JPj56phlFJS1Hr8t5zH3U773Ji5T6oPt4z6oPyq+Gzr/U6nIP88iWobqPIm+cE+eXNXvl43xLjVVcM8n3vEt+HGlaPldjitcROfnlza9kPhZ5vS6lqLdyTPmm5IB/n+5yT1hHlb3z03zZzV8PsmT1D0nq+lJ9Jy97F+/OwUepJ9bFOg32uXvlTYnWGzZDZQJrXO1qdN6i+DprlfZSPUXV+H7BtHzS3M1E+ztdm/aSyzndwP+eN/AYaR61TV3wrX9YH2Kluu/uf71TeknZy0u2C+jQJTfMcpVmzs7qXijzzc+n4zHN5722707qnvAepyDM/l/ZaZp/5+V5KP6punnsxDyojSrpzTL+oPnv55UdU/Ub55U9Q3YL88mavvKKkO8d0UH1G+eVH0HJfK+8zvpZq7wM8dpBPPh1U3lG+ZePWZH22g+Z4atA86B6EbhzlV7gOWo/ne1jaP/NcO+fnOqeodXg3coOWOSrv872Uguq3k0/e5Lwu8LGD/MrAmt/FNXgOzBzVhlF+ZSWobkF+ebNXfr9IunNMUXU5yicfDlqH5xbEo+rd7vQcz/YhqKyj/MrGtck+EwvKo9orU6vZysFxP8tP1DoFLMq/M/PcQTOj6hC1Du9D15Ju5znCaN+zSfXzWopdchml9wFy71QPG3TzauS9DygCXuN9gFxNh03aswT5WONzCPLNTzss9ne1xZ73e/moqezXYtL8eXym+WyFpPbKFK4U5TsiNGg574PMjq8838H5+S55xksdlf9Zn0sHLTMo7/O9lFpqcCyfPfKoxL70s61esfTYUT7vf1Q7RvmVk6C6BfnlzV75eV/HVtNhk16z0EetY5CyP6tnWc3poPJ5dlBZR/mVj9eS1QPc+PBICwfHrcEOSLGRGK+GYI2Ouffs7s8X/HmUr6D5GZJuF+TbaVx69dokv9m70gcALeFVXoPm+qgEm01/Jz/W4O7kK4pr14CXnK9u9KorrF9WQ10R5DfQvFab7dVPnNT+qiF7/qiyLDaw8r/1meKk+ZrcSdgsG7HyGiE6ahnvq1IshVee0Xs0/px2WmZQ3ud7LUWhBK/v1KByapgh87aX3/sHtWMU+fScIL+82asM76sbazhs0vLa6/2D1uFV3+/Uh6PK511UOR7vt5Wyk8VW9qQbz33pQcsq4ShfNvM3vfJ3bCTQe2+b+VLLeK5YmMTsWymTfASV4/2dPF+x5CWo7KDIQ6OYFUA7rK5I8uN5ReJZlI9R69QVUb5Xx/XAo72M2q4WrupztaUg3Xsp7pJlVN5L3S89hKqG5c5R7Q6IJKGU37WNJWbe38l38hPlp4YBS+Aa3suVPeuKQX4rOtbK9ygfo/oZkBxV3tKJpVYRoF9gS0H6JN+K5HPdJsh/pC1d8fcm+Rt0m/P+IC9L96PjOh5B+l9UVpLv4E+UX/3l1emfxIAb2pPkX1d4tb9v5SNpvT6TV7DX04DkpPJ92JKHydbCJv52wqu2FKQbzyXvt86kR/lKuq7SannJe5SvJGB93jNkHocxRbU/MwaU5t1+e5zuHOTX9q+Z3x7B3qT++i2l4wQbmArajkkE6BfbWpDuufTTvoi3VKLeS92vbUTey1/UbSPynnmdxB5W5JHkW7Zslqr0DJnXzNikfvZnYnusj+S5Dehblee54mbUOrxWASb1p9TtKw9taTbdBrG3fpr9xbYWpJ+vYvMSdT2vK0GM5dWo63gfQHM26HqeFSVL3ZHTpedK5GCdx0HlBPkdGJcEtOt8HZsXqytK93mifKy54ibIR49nbySVF7QNSQxiX2Ur96SfWQNkgXqUD5upvWam2TNAN7euPLBGPsrXtXkd5bsfPQnIZ9Q8Y+RVxq/9Pi4R5Yel7midfU9tRturrrB/u9Sqx0F+AVLSerwmGLxv8OhF0DZwoOqVthakG9tvEuXjvCTp0tn81pa6nyXN7+gZ9EZdl9eeAyKT+g/S3+jP5eGp3yvJ898u7TxD5rW0M96npPy83nEUW1bQPqsrrJ/ktWUkqlxd4fWOo9atK249mHipH4U1BPVvErPoV9tikG4jtD/Ix3lferrw70f5mXR7I3m+n9ljf9lDgy6fvfNq5ExS24Lmcm0/7fTyczl/I/9AHB/Zd8HzmiOrD5LyimLpJ7DUKL8A1tigeVJeQX0sdTdBaFnpW188JAEXOp7SnVO6NGiMTs93TnstE5X3+S5JB13mjfNzDmrHeXnbO82j6L/JN+9aS3v5Osj3/XMP2Pwo6c4hHdW2UX5lIqhuQX55s5efg/ze29qV3HXFXtKdQzpofbTDbaejyjhKunNK3tt3m7S1g+POPK9iu3TG1rtALz08Jsl/5CzqsoY+yldS3SwPv9PcuTje/9xpLqPMkrfFe8/0d8onyK/eZC86euNZpkscNhnlY+0VN6xWa19Q/5Jwta0G6aUOJXlK1GUV6pfy80HrXENRw4nll3TauXrtzx4G5jZKb1tEougMtC7Jt7HMuQVmkI9J7LVDf5J8b8PJ2S4P8gmMJq1fV9Am96Hnz9H7Zq1mbTVIr/0qtiDf68DWuoLl0qX9Ob294O9w9dpH1lC80zxjfg7M0RfPlUQ5TwO+5LueQxLQJ8/r2KLy1RVe5+XkWJ0QhB70HKT/Ktxkq0H6+So2L3Hhn+eWtI4a7kw/H2D2nCDfID2pDg+D850Yne/ZKN9Bynda3yC/zipL3dErG2jvra74Qn5tfhKwPZ7xVtO2GqQbz9mk15Zxec0ImaR1l197d2At2HxpyXuUn0l1NNo2q0Bwvh3n69i8RK3fSfaqM0dx7Rr6db6pxUvU+m2S1yz6qDx1RRBQN5a632jLQbrnvvSg5ytW+/0oP2s3yEn+X9Avb/yz3JJ8Bc17zm22hOB8W7y3olxyVsSlgvo5BAqojXdd8Z3WE8ThkqhTUL8m4SZbDtIn+RaceOXvl5Jj8MK7I/vSKeRRfjz3o9tswi9iz/lWec+QWflba2Aox5LYS0xi+Sr6N6mfwyajfAakkwhUsF3MpN9oy0G68Vzy/twMrudJ46PyfJk8Vy0Ya5SfWl5rvxfkJ8mHBTXMnmOUnzWvWIrywcwYtsL7OraodXgN6OUcEA0C0KWtB+lJfp5bchXlJ9egRZL/jNNTe1a9D4ybVN5e895zIMn3e7nGgOSgfq5SAmqV5FtXrBFcR1FXAGgIQbqfp2Z3o/xmNyflnfH2XLVgnhoU8Vy14LHU3QL0QcBHnjNkUcsHJbl2DSjDsw2PWj6o7lVXsOIGwE22HqR7XxEWH/2396nuOY3y3Zfy1JK5KD9JZdmd54OAP0ry/V4u2W8axFJ3oJRRvnXFoNsF+bV/SXmx3xfo1NaDdFPTVWxRfnJfyeQ9IGIezqZH+a5aSCrHlgqueUIu+uJ9Hdut30Ov/aWjOAQK2+N9daNNYtxaV0T5GJW/riBIBzpFkO4bOH7x6NdBPqZT+qD8PBt483Clgvd+9FKi2IOOl9khgl4dPet03zqAFOWDa9ewVZ7XsS05bNJrQI8VNwBuRpA+B6eeHdR4/+tBfkoFz0n1LHnfwn70oHkfOvCSGq5ju9Ygv0OgkoBt8l4Rd0u7HeVTVySx4gbAAgTpM88Oarz/+aX85Dww7jHv2fTznelRfpLKsNmDIOB1JeuAx275PnIIFOCjtcMmveqKUv3KSQC6RJA+K7HU+zkWnFvg6LX8OqlsJe99Z7o12IP8JJXJ7yAOisPlktq5YsnqyqjyJnGVEpDkW1dcs/ImyKcdnFSurmBPOtApgvSZZ+AYT+lH+Sm9isAGRJL82KzdD/JTaqm71x48tMt7hixc+HeXnAi/RBIA47n68Lwa7hJRPkrWpQTpQKc+Fcx5n1XU9iSVZyfqR21TUn5BzKLjeknzDFCQj0GvH3JonfOv5YOl7sBs1DzY7XVDih02ubvg73kNVieVM6k86zP/VVgTgy34E4L0j2yGM2pbLFieVN6oufH0auC9TCrTeNc4i/7hPv2qOR9+l3+jZJ1Mz1P+a2RnRnitNLEZ8tdOmr9mFm1No9j7CTxkdYVXW2N1xe6VvxPlM+A4qnxdMansu1odXEMbDmAj4indbSx5zUiZ92ozz5akvfJ7I+mugvSb5veNqncw5qA+y8AS9lnZZ+dVbl67ju3o9FxRfRvlk6+WguoW5Jc3e9XLu66Irzzfj07PFVTeLyr/np79R9zmKJ/vRBRuwp70j5K2NSo4yXcvvvcBch5K7EevoeG0GZbPTukbbe971Tr7rDxvYHjpiqUorl0DauF9deO7F/4syKctTPJZcfMflccqNCAzgvQ/+lnbkeQraXvBW1J+nve/T6f0lebZUALzdr2Xn6jnR93fygd70YGneR+6+1ygOMiH16DFB5X3pQBkRZD+R0nb4TkCfuZ9Z3pJSWVG2KN8TJoD9KR2bO1MhEudD9L08tQMWFD/VykBrUmqr64wHgN6k/zqiknl2QAJbSiQEUH6H21lCfakOoIpzxm70kosdY/yazQtQJ/UFpbrPc9z9tg62I/LcZSPJAAv8awr7AC5x3XFIJ9tMZ75kFSe5TttKJARQfofec8glVLLsv6t5LdJys+rwRzVXoAehZck+X03rfP3+AC5d/LBUnfgZUl+25usrhge/Z7XtpgkP5b/k8rzymtgEwjS/6zEjKe3mmawa1h2n9ukcveje2gxkGEG4HWe381vH/w6ajtXKQEtquWwySCfAdhR/nWFR9/V60pMYBMI0v8sqW92wMiketgWg94PGUsq43OVl9RmION5wF4rPL+b1vGL97/2mq3ZwgAisAYb+PeqK6I+1hVbXnHjcXjcU6ueAKyEIP3PkvoOGms7rM37GpcSSo1wB5XX4sqTIJa7X8L7OjbrcAf5HRiXBOAS3u34eeVNVHlJdQxUe52pxJJ3IBOC9Kf1fBVbUn16P7AvqYyg8jxG75fymm1pkfcM2Q/ywV504DreVzf+KJ82sJZJhkk+gwVBflfeAV37VHhKUp+jgzb4MKk+SXOw1+M+4aS+97W2tuokiA7FNezztXrDqz78WuVN4to14FqT5vYuqjxbdk1dMdfV36o8G0xtfetiOKWDfKRT+kbAIwTpT7PKZq/+1DxjbY1Lj0F6z0vdW8Qs+vVGbWtJYxKAW9gKlKjtqG3FjfXxPIJ0GySxtvV7tcsGeYJ8/CrgCSx3f1qPV4PZO42qV00nzq8pCbUYxCz6LZK2VY5Z6g7cJmlbdUVSXZL8ZrPtALmodnkMbpwlAU8gSH9eb1ex1b7vu8eBkUn9V76tXL8SxCz6ElsJXEdx7RqwRM9n+jw0qs66wnOPvK1AbaVP8ND5kFIPPfZ9sRKC9Ocl9aWFE9R7CwSS+hdUv6B5r1kQbpXU/1WJhmvXgGVGbaOuqLW/4jkhEzQf4NeSIN9r5HqbEMSK2JP+vKS5oWlxVPCxSW0EjHZ4XC95bkpWvl6dIrtvvOatCkF1BehRt593Yfv9PDu/dh1bz6sRJjGjASx1vrqx57oiqd4VN0l+B/hJH9u4Vg5Cs0EFzz5n77cbAdmMp3TXQdqrHRbwtZKvr6Wgsrzes9ZBlXBKR/mXg1bL02P2Of+mtvLsmjRou0b55XtQ3YL88mavNlldcddxGlS3Qf55tFf93sk/n4LKOcrnHaNwE5a7vyypD/9UO3oZVUwqP9I+ycc71SeKJe5rsxmyXpeDT+LaNWAtVlf0OkM4qf66YpT/loPhlH5RvW2w9Vt28lXrtcioBEH6y3poZD7cp1Yk9TE44rHPiFNdZ3ZKKwF6HqP6lARgTS1NDlyjlbNzash/u1bX2uJBdbF73XfyNwp4AUH6yyzoaSnAfUqLDWUPp8Mmlfcf+dnLPyiOmjsEvV7nVwOrD5P6w7VrwLqS+qwrktpg7WANB/gFzf2DH+XfRwiaZ/c9D4o7m8R+dLyCIP11rZ+8mNSeUW2fDjvJJ989B5SC/Gavg+ZOgP37Ucitt4B2FEsOgRx6m00f1U5dYX2o71WPrzXviba2OqosOyPh3f2//4XqwMAwXsXp7q+zka5v1aakNjuf5xUMUW1K8uG96iNobgR3KtMAxVN6q20f+OUhaS5rtXR2luLaNdQq6Lb6zfotNQx0n5+jlxtbWgusRs1tZFQ9hvs0aR7EScrXd4mab6AZVFcZnMQWK2A1rZ5oPKhdUW3ldQ35XtOJukflyYegeUT8IBV5j5pSUD1suWBLefdSOQWnu78kqI6y2mqe7tRW3j2XDmpTVP15e9Q8w26z7UsGf4PmfoftOa+5376Tj6N83jcKN/lEuITtpfla7flftb1s3CrZFkfgP5PfCoaD6qoQJ80jxnbOwAddly/22QfNjfbnmr+DQdvlWa4es8/mqPZnyOwu31EYNc+4eaipXD8lqL3BHOqK9bVcV9j+9JZWhJ5XU9rP6ZT+e//79uvw4O/9RR/7CUFtlLFJ8/fTg30Pg8r7SqwcuAnL3S9j+9JbC9JHtR2gG1sK9U5tSfLtHFlZjapH0MflbebhYYzTE3//3OCef6JO9jm2+P18aBIBOl7Xejvq7Xx1Y6vbBs2ktuuKneZl30FtsPY/qk/sRQdWVtMy4i0tL7EZ1Jby3HMZ01mLZZV0WQqqS+tlbS+cjfL7HILq19qWt6C6RLWVf4/ToPZFtZXnPaaDfB3l895RuAmnu1/m4exfCyb1sbSkxeueknz9XsEzYBtaL2vMaOBSzKYvk9R2XZHUvqR+765vxTcCrsBy98vZMuJWTjNO6oftZY5qw6Q68t6CjyjcYnzlzwfhoVbL2iiuXcPlrP0PwhLUFf7swE873yUKpdl1eJMAZBFVz5KZ11IrgwmXsCW1rSw13KseB7WRZ619fp7PF1Sng3zz5ZYUhYdG+X0WQfVr7TaDoDod1VY+1pyXtwpq83NoOe1Vh6N83j8KN2G5++XOJ03WblJbS/NfY3n+s9rwb9WDpbzXmUSe3aq1e8YnsSUE1+mpTfXUWl2R1N/s56T5tG22cJQxaZ5FB65GkH65Vval97jnaFQbkuqRRCByDdsrNgm3+EltdfgYjMG1Whmkr51dBdZSPrY2qHCpSQTqJUwin7EAQfp1WpjR/Un9Saq/kkuqL8izwJPG4XUWtCXhVufr2FowiWvXcL3WDo+tFXVFPaw8M8Obj5X1v4vBfyxAkH6d2gPgpH4rhNob9pqWup9NYtbwNfad3glLtTJDlgTchrp0HaPasIXPexQnjudi+crAHhYhSL/OpLqD4F6XZpn3qltSnSzfuHblaZPooKyllbMjCLRwqyRWJq1hUhuDZUnbMJ7SX0XZXovlo/UrelzVisII0q9Xc0c0qV8138k8qe6834kR3ccmsVdsbaPqNoqlh1iGQZ511J6Po7ZVV1j/wAL1SVjC+hPWrxgFrIAg/Xq1jo7Z4MGkvtU6QJJUt3PDMQlmEvmRQ1Ld34WeVxqhDFuZNAlLJdVdV2xxMGYS7eISk+aBDiZEsBqC9OvVesrrqP6NqjPva9yP/hiB+mwS+ZBTrZ3bSexHxzrYIrOOmgfdJ23TdEqfiS1y10qiX4EMCNKvV+Mpr5O2sf/F8r7G2bCkNkzadkMyiYY0t6Q6B9JYpoy1JFGe1jCqzrqCFTfSd+Ja0kvZgAb9CmRBkH6b2kaAk7ajtsGIpLYq50nzkqytHWpi78ueuzJqm4WZxB5BrGsnAvWlaryObRJ1xdmoOfhk0OJpk+b8+U4AqhJO6a6iFLUtR9WT9zu1a6d68jFnWrMR9XyPoDa8OaXf5JtXD9NeeM0oyvUtdqqjjLdcV9SUb4PwlEF19bu8005z2W3NUT75FQUU5lXYH6ejtmenOvK+h8rnC/Xb+B61/ufj+T5B7bADtu4qSUF4zSg+n1vZIGAN5bzVPP1R5Fsrdtp2sH5Q22XkKJ98iwIKq6UTWvv94TnUMvr+m/qxUx15uub3IsdIt+c7BbXDBn/uKkh74RKjKNdLBNUVvAS1I6qOPNsLlwjyrS880kF9BJpH+eRfFFBYVB2VR9A2HeSf973t6w5qv/E9aA4Qc/F8t6C22Gdx55yicIlRlOs1DKojWA9qy0HkWWuC5lUkR/l/djmSTcLs1Fe5OMonL6OAwmrYd3nQdn0t37y3NKhPQe0F6weVaQg83zGoLVG++XUULjWKcr2mQb6BZ1BbvNvzg7DEoDoGWtYqC4Pa3HP+mqN88jQKcGBfZs/KZNB21TBIEtS3oLmM/SLffH4pHVS2AfB816D2HOWXX4NwqVGU6xyC5nJo+65LtldB7fFszwdhDUHz7HrNfYbHycrd4f65ewzMHzrKJ4+jcJNPhCVs9Pdv8mNXwEzaruGUvpQPuz7me21H0MfyHuVr0nwtjO07L33P7kF+/q467xV+TpDfbPZ0Sp8JlxrkV5daPdpSuV7CggDbjhPuk/33/9P6WsvTIOqK3gTNfYVzn6GmAPjDKf1b85ZF+/VW6h/rM+Wob15jVy1+EACgiKh55NlmiHKPmlvnbX//7wWhFfaZ3TmlvQC0wrOuGIQSbHBq0Lxqp+RM+1FzP8X6D1H9z5ajI8ykA1iLNcLWAAb9cZbojZ5vGH/Xx1Hs/97/err/+eHRn6Mt1jkK8mEzY5MA1C7I9/wI6go/5z7D+edf9LHv8DA9Z3rw0/oJv+qP/YfzrwEAAKB5xsRrZuwnAWjFIL+6Yi8AAABgIzwPDooC0Iqj/OqKnNd1AgAAANWI8ut0HwWgFV/Lr674RQBQsf8RAADreSs//xCAVnjWFf8UAAAAsAFBfjNjd+L0f6AVQX71xFEAUDlm0gEAa/lafkZxSjPQinfykwQAAABshM1QMYsO4DXUFQAAAEBmg/w63QcBaMUgv7qCKxoBAACwGRYoe3W8BwFoxVF+dUUUAAAAsAFBfp3uowC0Ioq6AgBexcFxAICl3snPvwSgFW/lhysaAQAAsAlBfjNjd+IQKKAVQdQVAHARZtIBAEtE+RnFtWtAK76Vn1HUFQAAANiIo/xmxqIAtMKzrggCAAAANuBr+XW6jwLQikF+dcVBAAAAwEZY59er4z0IQCuoKwAAAIDMgvw63cdTeiMALfhCvnUFADTnUwEAerRXXl/ITzql3wVgDT3XFVzRCKBJnwgA0KM79eszcVIzsBbqCgCoDFewAQBakkSnG8DrRlFXAGgUQToAoCX/FAC8jqXuAJrFcncA6FOPS1gnzctXAaynx7oindJXAoBGMZMOAGjFPwQAr2MWHUDTmEkHgD71Njs2iVl0IAfqCgCoDDPpAIAWMIsO4BLUFQCax0w6APSpp9mxScyMAbn0VFcksRcdQAeYSQcA1O57AcDrvhEAAABQqbtO0l4AcmqlLqCuAAAAQNNa6Vi/lH47pSAAObVUJzyXjqKuAAAAQOVa6Vy/lL4TgNxaqhOeS4MAAACAyrXUwX4qHQSghFbqhOfSXgAAAEADWulgP5WOYukqUEor9QJ1BQAAAJrWSif7qRQFoJSW6obHKQgAAABoREsd7YdpJwAltVQ/PEycWQEAAICmtNTZJkAH/LRUR1BXAAAAoFktdbjpdAN+WqonqCsAAADQLDrdAC5BXQEAAAAU0EKH+zdxvzHgrZW6IgoAAABoWO2d7h/FycxADWqvK96f0hsBAAAAjau1w23BeRSAWtRYT9jMuQXnQQAAAEAnaupsHzRfl8RsGFAf6goAqMynAgBguen+54dT+vX+vz/cp98FALNJc51gP891RdJcVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGCD/j+MEQEBc5LsYAAAAABJRU5ErkJggg=="],["x",0.05],["xanchor","center"],["xref","paper"],["y",1.12],["yanchor","middle"],["yref","paper"]]}]],["paper_bgcolor","#FFFFFF"],["plot_bgcolor","#FFFFFF"],["width",800],["showlegend",false],["shapes",[{"type":"map","entries":[["line",{"type":"map","entries":[["color","#736e6e"],["dash","dash"]]}],["type","line"],["x0",0],["x1",1],["xref","x domain"],["y0",5.174825174825175],["y1",5.174825174825175],["yref","y"]]}]],["annotations",[{"type":"map","entries":[["showarrow",false],["text","Overall: 5.2%"],["x",1],["xanchor","right"],["xref","x domain"],["y",5.174825174825175],["yanchor","bottom"],["yref","y"]]}]]]},"data_sources":[{"type":"object","name":"ColumnDataSource","id":"1f8c2016-7591-487e-ac31-1e229a4b20e5","attributes":{"selected":{"type":"object","name":"Selection","id":"49868529-55d4-4de3-a4d2-da69045fb447","attributes":{"indices":[],"line_indices":[]}},"selection_policy":{"type":"object","name":"UnionRenderers","id":"779dfc9b-0c34-4129-8a87-695a99a5bede"},"data":{"type":"map","entries":[["text",[{"type":"ndarray","array":{"type":"bytes","data":"H4sIAAEAAAAC/+O6vrjAdpWDg2ZM/6GvK7Qc1rk/rBLRU4LTtlwgBYoOSwpALHGHNDBgczAGg8/2DGQCAJD/evh4AAAA"},"shape":[15],"dtype":"float64","order":"little"}]],["x",[{"type":"ndarray","array":["investment holding companies","food & drug retailing","software & it services","industrial & commercial services","personal & household products & services","cyclical consumer products","food & beverages","retailers","chemicals","banking & investment services","automobiles & auto parts","applied resources","industrial goods","healthcare services & equipment","cyclical consumer services"],"shape":[15],"dtype":"object","order":"little"}]],["y",[{"type":"ndarray","array":{"type":"bytes","data":"H4sIAAEAAAAC/+O6vrjAdpWDg2ZM/6GvK7Qc1rk/rBLRU4LTtlwgBYoOSwpALHGHNDBgczAGg8/2DGQCAJD/evh4AAAA"},"shape":[15],"dtype":"float64","order":"little"}]]]}}}],"relayout":null,"restyle":null,"viewport_update_throttle":200}},{"type":"object","name":"panel.models.comm_manager.CommManager","id":"9d8b8e9a-dc8c-497f-bb59-3d2ef6a7acd9","attributes":{"plot_id":"db62437a-60ea-4359-99f3-e173fedf9cf6","comm_id":"15b8aa159fe0444e9f93a7c716b338d7","client_comm_id":"ff21ff680ec8481b9ab2c6df5db0a918"}}],"defs":[{"type":"model","name":"ReactiveHTML1"},{"type":"model","name":"FlexBox1","properties":[{"name":"align_content","kind":"Any","default":"flex-start"},{"name":"align_items","kind":"Any","default":"flex-start"},{"name":"flex_direction","kind":"Any","default":"row"},{"name":"flex_wrap","kind":"Any","default":"wrap"},{"name":"gap","kind":"Any","default":""},{"name":"justify_content","kind":"Any","default":"flex-start"}]},{"type":"model","name":"FloatPanel1","properties":[{"name":"config","kind":"Any","default":{"type":"map"}},{"name":"contained","kind":"Any","default":true},{"name":"position","kind":"Any","default":"right-top"},{"name":"offsetx","kind":"Any","default":null},{"name":"offsety","kind":"Any","default":null},{"name":"theme","kind":"Any","default":"primary"},{"name":"status","kind":"Any","default":"normalized"}]},{"type":"model","name":"GridStack1","properties":[{"name":"ncols","kind":"Any","default":null},{"name":"nrows","kind":"Any","default":null},{"name":"allow_resize","kind":"Any","default":true},{"name":"allow_drag","kind":"Any","default":true},{"name":"state","kind":"Any","default":[]}]},{"type":"model","name":"drag1","properties":[{"name":"slider_width","kind":"Any","default":5},{"name":"slider_color","kind":"Any","default":"black"},{"name":"start","kind":"Any","default":0},{"name":"end","kind":"Any","default":100},{"name":"value","kind":"Any","default":50}]},{"type":"model","name":"click1","properties":[{"name":"terminal_output","kind":"Any","default":""},{"name":"debug_name","kind":"Any","default":""},{"name":"clears","kind":"Any","default":0}]},{"type":"model","name":"ReactiveESM1","properties":[{"name":"esm_constants","kind":"Any","default":{"type":"map"}}]},{"type":"model","name":"JSComponent1","properties":[{"name":"esm_constants","kind":"Any","default":{"type":"map"}}]},{"type":"model","name":"ReactComponent1","properties":[{"name":"use_shadow_dom","kind":"Any","default":true},{"name":"esm_constants","kind":"Any","default":{"type":"map"}}]},{"type":"model","name":"AnyWidgetComponent1","properties":[{"name":"use_shadow_dom","kind":"Any","default":true},{"name":"esm_constants","kind":"Any","default":{"type":"map"}}]},{"type":"model","name":"FastWrapper1","properties":[{"name":"object","kind":"Any","default":null},{"name":"style","kind":"Any","default":null}]},{"type":"model","name":"NotificationArea1","properties":[{"name":"js_events","kind":"Any","default":{"type":"map"}},{"name":"max_notifications","kind":"Any","default":5},{"name":"notifications","kind":"Any","default":[]},{"name":"position","kind":"Any","default":"bottom-right"},{"name":"_clear","kind":"Any","default":0},{"name":"types","kind":"Any","default":[{"type":"map","entries":[["type","warning"],["background","#ffc107"],["icon",{"type":"map","entries":[["className","fas fa-exclamation-triangle"],["tagName","i"],["color","white"]]}]]},{"type":"map","entries":[["type","info"],["background","#007bff"],["icon",{"type":"map","entries":[["className","fas fa-info-circle"],["tagName","i"],["color","white"]]}]]}]}]},{"type":"model","name":"Notification","properties":[{"name":"background","kind":"Any","default":null},{"name":"duration","kind":"Any","default":3000},{"name":"icon","kind":"Any","default":null},{"name":"message","kind":"Any","default":""},{"name":"notification_type","kind":"Any","default":null},{"name":"_rendered","kind":"Any","default":false},{"name":"_destroyed","kind":"Any","default":false}]},{"type":"model","name":"TemplateActions1","properties":[{"name":"open_modal","kind":"Any","default":0},{"name":"close_modal","kind":"Any","default":0}]},{"type":"model","name":"BootstrapTemplateActions1","properties":[{"name":"open_modal","kind":"Any","default":0},{"name":"close_modal","kind":"Any","default":0}]},{"type":"model","name":"TemplateEditor1","properties":[{"name":"layout","kind":"Any","default":[]}]},{"type":"model","name":"MaterialTemplateActions1","properties":[{"name":"open_modal","kind":"Any","default":0},{"name":"close_modal","kind":"Any","default":0}]},{"type":"model","name":"request_value1","properties":[{"name":"fill","kind":"Any","default":"none"},{"name":"_synced","kind":"Any","default":null},{"name":"_request_sync","kind":"Any","default":0}]}]}}; var render_items = [{"docid":"bf743a33-d651-471f-bac1-85b23c493483","roots":{"db62437a-60ea-4359-99f3-e173fedf9cf6":"c8a3366c-6142-4880-9930-7a4ab958dbf5"},"root_ids":["db62437a-60ea-4359-99f3-e173fedf9cf6"]}]; var docs = Object.values(docs_json) if (!docs) { return } const version = docs[0].version.replace('rc', '-rc.').replace('.dev', '-dev.') async function embed_document(root) { var Bokeh = get_bokeh(root) await Bokeh.embed.embed_items_notebook(docs_json, render_items); for (const render_item of render_items) { for (const root_id of render_item.root_ids) { const id_el = document.getElementById(root_id) if (id_el.children.length && id_el.children[0].hasAttribute('data-root-id')) { const root_el = id_el.children[0] root_el.id = root_el.id + '-rendered' for (const child of root_el.children) { // Ensure JupyterLab does not capture keyboard shortcuts // see: https://jupyterlab.readthedocs.io/en/4.1.x/extension/notebook.html#keyboard-interaction-model child.setAttribute('data-lm-suppress-shortcuts', 'true') } } } } } function get_bokeh(root) { if (root.Bokeh === undefined) { return null } else if (root.Bokeh.version !== version) { if (root.Bokeh.versions === undefined || !root.Bokeh.versions.has(version)) { return null } return root.Bokeh.versions.get(version); } else if (root.Bokeh.version === version) { return root.Bokeh } return null } function is_loaded(root) { var Bokeh = get_bokeh(root) return (Bokeh != null && Bokeh.Panel !== undefined && ( root.Plotly !== undefined) && ( root.Plotly !== undefined)) } if (is_loaded(root)) { embed_document(root); } else { var attempts = 0; var timer = setInterval(function(root) { if (is_loaded(root)) { clearInterval(timer); embed_document(root); } else if (document.readyState == "complete") { attempts++; if (attempts > 200) { clearInterval(timer); var Bokeh = get_bokeh(root) if (Bokeh == null || Bokeh.Panel == null) { console.warn("Panel: ERROR: Unable to run Panel code because Bokeh or Panel library is missing"); } else { console.warn("Panel: WARNING: Attempting to render but not all required libraries could be resolved.") embed_document(root) } } } }, 25, root) } })(window);

By UW Channel (Deal Size Tier)

By Platform

Cross-Tabulation: Segment x Channel# This heatmap combines segment and sales channel into a single view, showing where bad prediction rates are highest when both dimensions are considered together. Darker cells indicate higher UTP rates among grade A/B deals.

By Region

By Segment (New Business vs Customer Success)

Statistical Methodology

Statistical Methodology

By Segment (New Business vs Customer Success)

Monthly Rank Ordering Across All Grades

Monthly Rank Ordering Across All Grades

Monthly UTP Rate for Grade A/B Deals

Monthly UTP Rate for Grade A/B Deals

Key Observations New Business grade B warrants further investigation. At 18.8% UTP (9 of 48 deals), NB grade B performs worse than grades C (9.0%) and D (9.4%). The model labels these customers as “low risk” but almost 1 in 5 default. By contrast, CS grade B is at just 0.9% (1 of 106). The B grade does not appear to provide meaningful risk separation for New Business. D/E-grade UTP reflects UW decisions. Grade E deals are UW overrides; grade D deals are where the model defers to underwriter judgement. The 15-19% UTP rates at D/E should be interpreted accordingly.

By Segment (New Business vs Customer Success)

The out-of-time (OOT) sample covers funded deals from 21 Aug 2024 to 5 Feb 2025 – a period the model never saw during training. The table below summarises the overall population and the subset of “bad predictions” (grade A/B deals that ultimately went UTP within 365 days).

What Is a “Bad Prediction”?# A bad prediction (false negative) is a deal where: The model assigned a low risk grade (A or B) The customer ended up UTP within 365 days These are the cases where the model got it most wrong – it signalled low risk, the customer was funded, and they defaulted.

This analysis is specifically focused on funded deals only. Every deal in this sample was approved and received money, regardless of what the model recommended.

How Model Grades Work# The delinquency model produces a continuous probability score for each deal, estimating the likelihood the customer will become UTP within 365 days. This score is then mapped to a credit grade from A to E, where A is the lowest risk and E is the highest risk. Grades vs Underwriting Decisions# An important distinction: the model grade is a recommendation, not the final decision. The underwriting (UW) team reviews each deal and can approve or decline regardless of the model grade. This means: A customer graded A or B by the model can still go UTP – this is a genuine model failure (a “bad prediction”). The model said they were low risk, UW agreed and funded them, and the customer defaulted. A customer graded D by the model may still be funded. The model cannot reliably separate goods from bads in this band, so underwriters use their own judgement to make the call. A customer graded E by the model who appears in the funded data represents a UW override – the model flagged the highest risk, and the UW team chose to approve anyway.

Context

Key Questions

Overall

By Sales Channel

By Segment (New Business vs Customer Success)

Monthly UTP Rate for Grade A/B Deals

By Segment (New Business vs Customer Success)

Escalations Report Q1-25

5.1 Win Rate Comparison: Escalated vs. Non-Escalated Deals