GIT c68d50ca6e55d067a3116bfa1912db38da33aa48 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git#for-mm commit cb8d4a372684793ff2f1eb35f2fdab60a96aff16 Author: Roland Dreier Date: Mon Nov 20 13:05:19 2006 -0800 IPoIB: Convert to NAPI Convert the IP-over-InfiniBand network device driver over to using NAPI to handle all completions (both receive and send). Signed-off-by: Roland Dreier commit 5f96e742c413206ffc65c381fe4735918b83319a Author: Roland Dreier Date: Mon Nov 20 13:05:19 2006 -0800 IB: Return "maybe missed event" hint from ib_req_notify_cq() The semantics defined by the InfiniBand specification say that completion events are only generated when a completions is added to a completion queue (CQ) after completion notification is requested. In other words, this means that the following race is possible: while (CQ is not empty) ib_poll_cq(CQ); // new completion is added after while loop is exited ib_req_notify_cq(CQ); // no event is generated for the existing completion To close this race, the IB spec recommends doing another poll of the CQ after requesting notification. However, it is not always possible to arrange code this way (for example, we have found that NAPI for IPoIB cannot poll after requesting notification). Also, some hardware (eg Mellanox HCAs) actually will generate an event for completions added before the call to ib_req_notify_cq() -- which is allowed by the spec, since there's no way for any upper-layer consumer to know exactly when a completion was really added -- so the extra poll of the CQ is just a waste. Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for ib_req_notify_cq() so that it can return a hint about whether the a completion may have been added before the request for notification. The return value of ib_req_notify_cq() is extended so: < 0 means an error occurred while requesting notification == 0 means notification was requested successfully, and if IB_CQ_REPORT_MISSED_EVENTS was passed in, then no events were missed and it is safe to wait for another event. > 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was passed in. It means that the consumer must poll the CQ again to make sure it is empty to avoid the race described above. We add a flag to enable this behavior rather than turning it on uncondiationally, because checking for missed events may incur significant overhead for some low-level drivers, and consumers that don't care about the results of this test shouldn't be forced to pay for the test. Signed-off-by: Roland Dreier commit 6e28e98737494c24ea3114c1b6cf518eb26e10cd Author: Roland Dreier Date: Mon Nov 20 13:05:03 2006 -0800 RDMA/amso1100: Fix section mismatches The amso1100 driver was missing a couple of __devinit/__devexit annotations for init/cleanup functions that are called from __devinit/__devexit functions. Reported by Randy Dunlap . Signed-off-by: Roland Dreier commit e98be10e1e4667a57eb212c9b6cd604e16cd711c Author: Roland Dreier Date: Mon Nov 20 13:05:03 2006 -0800 IB/mthca: Fix section mismatches Commit b3b30f5e ("IB/mthca: Recover from catastrophic errors") introduced some section mismatch breakage, because the error recovery code tears down and reinitializes the device, which calls into lots of code originally marked __devinit and __devexit from regular .text. Fix this by getting rid of these now-incorrect section markers. Reported by Randy Dunlap . Signed-off-by: Roland Dreier commit b0730bd56e74d891a9b5a4fce45549fefc12c06e Author: Arne Redlich Date: Wed Nov 15 12:43:00 2006 +0100 IB/srp: Increase supported CDB size Set the Scsi_Host's max_cmd_len from 12 (default) to 16 for SRP. Otherwise scsi_dispatch_cmd() won't pass down certain commands such as READ CAPACITY 16, required for supporting disks > 2TB. Signed-off-by: Arne Redlich Signed-off-by: Roland Dreier commit 8a7eb2d699a725e003a27df623c2d535c6f637aa Author: Dotan Barak Date: Tue Oct 24 13:35:27 2006 -0700 RDMA/cm: Remove setting local write as part of QP access flags The qp_access_flags are for remote access permissions only, so IB_ACCESS_LOCAL_WRITE is an invalid value. Remove it from the values set by cm_init_qp_init_attr() and cma_init_ib_qp(). Signed-off-by: Dotan Barak Signed-off-by: Sean Hefty Signed-off-by: Roland Dreier commit fc89f196ee05f6b3ed3d5b25ed16cf384898de38 Author: Eric Sesterhenn Date: Mon Oct 23 22:17:21 2006 +0200 IB: kmemdup() cleanup Replace open coded kmemdup() to save some screen space, and allow inlining/not inlining to be triggered by gcc. Signed-off-by: Eric Sesterhenn Signed-off-by: Roland Dreier commit e5b5b38b96cb1edb320ac8353b5f26a0d33b5e9a Author: Krishna Kumar Date: Tue Oct 17 10:09:11 2006 +0530 RDMA/cma: Rewrite cma_req_handler() to encapsulate common code Rewrite cma_req_handler error handling case to encapsulate common code. Signed-off-by: Krishna Kumar Acked-by: Sean Hefty Signed-off-by: Roland Dreier commit 1f82b602adb9231bac9e4185fca90bb278b76d2f Author: Krishna Kumar Date: Tue Oct 17 10:09:09 2006 +0530 RDMA/addr: Use time_after_eq() instead of time_after() in queue_req() In queue_req(), use time_after_eq() instead of time_after() for following reasons : - Improves insert time if multiple entries with same time are present. - set_timeout need not be called if entry with same time is added to the list (and that happens to be the entry with the smallest time), saving atomic/locking operations. - Earlier entries with same time are deleted first (fifo). Signed-off-by: Krishna Kumar Acked-by: Sean Hefty Signed-off-by: Roland Dreier commit cb57669daf579cf7d433f66c07163c2d36c88f23 Author: Krishna Kumar Date: Mon Oct 16 10:09:08 2006 +0530 RDMA/cma: Remove redundant check in cma_add_one Remove redundant check of node_guid in cma_add_one(). Signed-off-by: Krishna Kumar Acked-by: Sean Hefty Signed-off-by: Roland Dreier commit 663bf4cc0c45f1f048369b0b6b6b7f6f1d1051d7 Author: Krishna Kumar Date: Mon Oct 16 10:09:01 2006 +0530 RDMA/cma: Optimize cma_bind_loopback() to check for empty list Optimize to test for an empty list first. This ends up simplifying the code too. Signed-off-by: Krishna Kumar Acked-by: Sean Hefty Signed-off-by: Roland Dreier drivers/infiniband/core/addr.c | 2 - drivers/infiniband/core/cm.c | 6 +- drivers/infiniband/core/cma.c | 49 +++++++--------- drivers/infiniband/core/iwcm.c | 3 - drivers/infiniband/core/ucm.c | 6 +- drivers/infiniband/hw/amso1100/c2.h | 2 - drivers/infiniband/hw/amso1100/c2_cq.c | 16 ++++- drivers/infiniband/hw/amso1100/c2_rnic.c | 4 + drivers/infiniband/hw/ehca/ehca_iverbs.h | 2 - drivers/infiniband/hw/ehca/ehca_reqs.c | 14 ++++- drivers/infiniband/hw/ehca/ipz_pt_fn.h | 8 +++ drivers/infiniband/hw/ipath/ipath_cq.c | 15 ++++- drivers/infiniband/hw/ipath/ipath_verbs.h | 2 - drivers/infiniband/hw/mthca/mthca_av.c | 3 - drivers/infiniband/hw/mthca/mthca_cq.c | 15 +++-- drivers/infiniband/hw/mthca/mthca_dev.h | 4 + drivers/infiniband/hw/mthca/mthca_eq.c | 21 +++---- drivers/infiniband/hw/mthca/mthca_mad.c | 2 - drivers/infiniband/hw/mthca/mthca_main.c | 29 +++++----- drivers/infiniband/hw/mthca/mthca_mcg.c | 3 - drivers/infiniband/hw/mthca/mthca_mr.c | 5 +- drivers/infiniband/hw/mthca/mthca_pd.c | 3 - drivers/infiniband/hw/mthca/mthca_provider.c | 3 - drivers/infiniband/hw/mthca/mthca_qp.c | 3 - drivers/infiniband/hw/mthca/mthca_srq.c | 2 - drivers/infiniband/ulp/ipoib/ipoib.h | 1 drivers/infiniband/ulp/ipoib/ipoib_ib.c | 78 +++++++++++++++++++++----- drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 + drivers/infiniband/ulp/srp/ib_srp.c | 3 + include/rdma/ib_verbs.h | 40 ++++++++++--- 30 files changed, 215 insertions(+), 131 deletions(-) diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index e11187e..a68d7c7 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c @@ -139,7 +139,7 @@ static void queue_req(struct addr_req *r mutex_lock(&lock); list_for_each_entry_reverse(temp_req, &req_list, list) { - if (time_after(req->timeout, temp_req->timeout)) + if (time_after_eq(req->timeout, temp_req->timeout)) break; } diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index 25b1018..78d9c0c 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -240,11 +240,10 @@ static void * cm_copy_private_data(const if (!private_data || !private_data_len) return NULL; - data = kmalloc(private_data_len, GFP_KERNEL); + data = kmemdup(private_data, private_data_len, GFP_KERNEL); if (!data) return ERR_PTR(-ENOMEM); - memcpy(data, private_data, private_data_len); return data; } @@ -3173,8 +3172,7 @@ static int cm_init_qp_init_attr(struct c case IB_CM_ESTABLISHED: *qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS | IB_QP_PKEY_INDEX | IB_QP_PORT; - qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE | - IB_ACCESS_REMOTE_WRITE; + qp_attr->qp_access_flags = IB_ACCESS_REMOTE_WRITE; if (cm_id_priv->responder_resources) qp_attr->qp_access_flags |= IB_ACCESS_REMOTE_READ | IB_ACCESS_REMOTE_ATOMIC; diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 845090b..cf48f26 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -344,7 +344,7 @@ static int cma_init_ib_qp(struct rdma_id return ret; qp_attr.qp_state = IB_QPS_INIT; - qp_attr.qp_access_flags = IB_ACCESS_LOCAL_WRITE; + qp_attr.qp_access_flags = 0; qp_attr.port_num = id_priv->id.port_num; return ib_modify_qp(qp, &qp_attr, IB_QP_STATE | IB_QP_ACCESS_FLAGS | IB_QP_PKEY_INDEX | IB_QP_PORT); @@ -935,13 +935,8 @@ static int cma_req_handler(struct ib_cm_ mutex_lock(&lock); ret = cma_acquire_dev(conn_id); mutex_unlock(&lock); - if (ret) { - ret = -ENODEV; - cma_exch(conn_id, CMA_DESTROYING); - cma_release_remove(conn_id); - rdma_destroy_id(&conn_id->id); - goto out; - } + if (ret) + goto release_conn_id; conn_id->cm_id.ib = cm_id; cm_id->context = conn_id; @@ -951,13 +946,17 @@ static int cma_req_handler(struct ib_cm_ ret = cma_notify_user(conn_id, RDMA_CM_EVENT_CONNECT_REQUEST, 0, ib_event->private_data + offset, IB_CM_REQ_PRIVATE_DATA_SIZE - offset); - if (ret) { - /* Destroy the CM ID by returning a non-zero value. */ - conn_id->cm_id.ib = NULL; - cma_exch(conn_id, CMA_DESTROYING); - cma_release_remove(conn_id); - rdma_destroy_id(&conn_id->id); - } + if (!ret) + goto out; + + /* Destroy the CM ID by returning a non-zero value. */ + conn_id->cm_id.ib = NULL; + +release_conn_id: + cma_exch(conn_id, CMA_DESTROYING); + cma_release_remove(conn_id); + rdma_destroy_id(&conn_id->id); + out: cma_release_remove(listen_id); return ret; @@ -1481,19 +1480,18 @@ static int cma_bind_loopback(struct rdma u8 p; mutex_lock(&lock); + if (list_empty(&dev_list)) { + ret = -ENODEV; + goto out; + } list_for_each_entry(cma_dev, &dev_list, list) for (p = 1; p <= cma_dev->device->phys_port_cnt; ++p) - if (!ib_query_port (cma_dev->device, p, &port_attr) && + if (!ib_query_port(cma_dev->device, p, &port_attr) && port_attr.state == IB_PORT_ACTIVE) goto port_found; - if (!list_empty(&dev_list)) { - p = 1; - cma_dev = list_entry(dev_list.next, struct cma_device, list); - } else { - ret = -ENODEV; - goto out; - } + p = 1; + cma_dev = list_entry(dev_list.next, struct cma_device, list); port_found: ret = ib_get_cached_gid(cma_dev->device, p, 0, &gid); @@ -2123,8 +2121,6 @@ static void cma_add_one(struct ib_device cma_dev->device = device; cma_dev->node_guid = device->node_guid; - if (!cma_dev->node_guid) - goto err; init_completion(&cma_dev->comp); atomic_set(&cma_dev->refcount, 1); @@ -2136,9 +2132,6 @@ static void cma_add_one(struct ib_device list_for_each_entry(id_priv, &listen_any_list, list) cma_listen_on_dev(id_priv, cma_dev); mutex_unlock(&lock); - return; -err: - kfree(cma_dev); } static int cma_remove_id_dev(struct rdma_id_private *id_priv) diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index c3fb304..2bbcfa5 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -140,10 +140,9 @@ static int copy_private_data(struct iwcm { void *p; - p = kmalloc(event->private_data_len, GFP_ATOMIC); + p = kmemdup(event->private_data, event->private_data_len, GFP_ATOMIC); if (!p) return -ENOMEM; - memcpy(p, event->private_data, event->private_data_len); event->private_data = p; return 0; } diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c index ad4f4d5..b4894ba 100644 --- a/drivers/infiniband/core/ucm.c +++ b/drivers/infiniband/core/ucm.c @@ -328,20 +328,18 @@ static int ib_ucm_event_process(struct i } if (uvt->data_len) { - uvt->data = kmalloc(uvt->data_len, GFP_KERNEL); + uvt->data = kmemdup(evt->private_data, uvt->data_len, GFP_KERNEL); if (!uvt->data) goto err1; - memcpy(uvt->data, evt->private_data, uvt->data_len); uvt->resp.present |= IB_UCM_PRES_DATA; } if (uvt->info_len) { - uvt->info = kmalloc(uvt->info_len, GFP_KERNEL); + uvt->info = kmemdup(info, uvt->info_len, GFP_KERNEL); if (!uvt->info) goto err2; - memcpy(uvt->info, info, uvt->info_len); uvt->resp.present |= IB_UCM_PRES_INFO; } return 0; diff --git a/drivers/infiniband/hw/amso1100/c2.h b/drivers/infiniband/hw/amso1100/c2.h index 1b17dcd..a74a3b8 100644 --- a/drivers/infiniband/hw/amso1100/c2.h +++ b/drivers/infiniband/hw/amso1100/c2.h @@ -519,7 +519,7 @@ extern void c2_free_cq(struct c2_dev *c2 extern void c2_cq_event(struct c2_dev *c2dev, u32 mq_index); extern void c2_cq_clean(struct c2_dev *c2dev, struct c2_qp *qp, u32 mq_index); extern int c2_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *entry); -extern int c2_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify notify); +extern int c2_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); /* CM */ extern int c2_llp_connect(struct iw_cm_id *cm_id, diff --git a/drivers/infiniband/hw/amso1100/c2_cq.c b/drivers/infiniband/hw/amso1100/c2_cq.c index 05c9154..2d8a295 100644 --- a/drivers/infiniband/hw/amso1100/c2_cq.c +++ b/drivers/infiniband/hw/amso1100/c2_cq.c @@ -217,17 +217,19 @@ int c2_poll_cq(struct ib_cq *ibcq, int n return npolled; } -int c2_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify notify) +int c2_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags notify_flags) { struct c2_mq_shared __iomem *shared; struct c2_cq *cq; + unsigned long flags; + int ret = 0; cq = to_c2cq(ibcq); shared = cq->mq.peer; - if (notify == IB_CQ_NEXT_COMP) + if ((notify_flags & IB_CQ_SOLICITED_MASK) == IB_CQ_NEXT_COMP) writeb(C2_CQ_NOTIFICATION_TYPE_NEXT, &shared->notification_type); - else if (notify == IB_CQ_SOLICITED) + else if ((notify_flags & IB_CQ_SOLICITED_MASK) == IB_CQ_SOLICITED) writeb(C2_CQ_NOTIFICATION_TYPE_NEXT_SE, &shared->notification_type); else return -EINVAL; @@ -241,7 +243,13 @@ int c2_arm_cq(struct ib_cq *ibcq, enum i */ readb(&shared->armed); - return 0; + if (notify_flags & IB_CQ_REPORT_MISSED_EVENTS) { + spin_lock_irqsave(&cq->lock, flags); + ret = !c2_mq_empty(&cq->mq); + spin_unlock_irqrestore(&cq->lock, flags); + } + + return ret; } static void c2_free_cq_buf(struct c2_dev *c2dev, struct c2_mq *mq) diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c b/drivers/infiniband/hw/amso1100/c2_rnic.c index 623dc95..1687c51 100644 --- a/drivers/infiniband/hw/amso1100/c2_rnic.c +++ b/drivers/infiniband/hw/amso1100/c2_rnic.c @@ -441,7 +441,7 @@ static int c2_rnic_close(struct c2_dev * * involves initalizing the various limits and resouce pools that * comprise the RNIC instance. */ -int c2_rnic_init(struct c2_dev *c2dev) +int __devinit c2_rnic_init(struct c2_dev *c2dev) { int err; u32 qsize, msgsize; @@ -611,7 +611,7 @@ int c2_rnic_init(struct c2_dev *c2dev) /* * Called by c2_remove to cleanup the RNIC resources. */ -void c2_rnic_term(struct c2_dev *c2dev) +void __devexit c2_rnic_term(struct c2_dev *c2dev) { /* Close the open adapter instance */ diff --git a/drivers/infiniband/hw/ehca/ehca_iverbs.h b/drivers/infiniband/hw/ehca/ehca_iverbs.h index 3720e30..313bc9c 100644 --- a/drivers/infiniband/hw/ehca/ehca_iverbs.h +++ b/drivers/infiniband/hw/ehca/ehca_iverbs.h @@ -135,7 +135,7 @@ int ehca_poll_cq(struct ib_cq *cq, int n int ehca_peek_cq(struct ib_cq *cq, int wc_cnt); -int ehca_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify cq_notify); +int ehca_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify_flags notify_flags); struct ib_qp *ehca_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *init_attr, diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c index b46bda1..f55742c 100644 --- a/drivers/infiniband/hw/ehca/ehca_reqs.c +++ b/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -634,11 +634,13 @@ poll_cq_exit0: return ret; } -int ehca_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify cq_notify) +int ehca_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify_flags notify_flags) { struct ehca_cq *my_cq = container_of(cq, struct ehca_cq, ib_cq); + unsigned long spl_flags; + int ret = 0; - switch (cq_notify) { + switch (notify_flags & IB_CQ_SOLICITED_MASK) { case IB_CQ_SOLICITED: hipz_set_cqx_n0(my_cq, 1); break; @@ -649,5 +651,11 @@ int ehca_req_notify_cq(struct ib_cq *cq, return -EINVAL; } - return 0; + if (notify_flags & IB_CQ_REPORT_MISSED_EVENTS) { + spin_lock_irqsave(&my_cq->spinlock, spl_flags); + ret = ipz_qeit_is_valid(&my_cq->ipz_queue); + spin_unlock_irqrestore(&my_cq->spinlock, spl_flags); + } + + return ret; } diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.h b/drivers/infiniband/hw/ehca/ipz_pt_fn.h index 2f13509..5601755 100644 --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.h +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.h @@ -140,6 +140,14 @@ static inline void *ipz_qeit_get_inc_val return cqe; } +static inline int ipz_qeit_is_valid(struct ipz_queue *queue) +{ + struct ehca_cqe *cqe = ipz_qeit_get(queue); + u32 cqe_flags = cqe->cqe_flags; + + return cqe_flags >> 7 == (queue->toggle_state & 1); +} + /* * returns and resets Queue Entry iterator * returns address (kv) of first Queue Entry diff --git a/drivers/infiniband/hw/ipath/ipath_cq.c b/drivers/infiniband/hw/ipath/ipath_cq.c index 87462e0..9582145 100644 --- a/drivers/infiniband/hw/ipath/ipath_cq.c +++ b/drivers/infiniband/hw/ipath/ipath_cq.c @@ -306,17 +306,18 @@ int ipath_destroy_cq(struct ib_cq *ibcq) /** * ipath_req_notify_cq - change the notification type for a completion queue * @ibcq: the completion queue - * @notify: the type of notification to request + * @notify_flags: the type of notification to request * * Returns 0 for success. * * This may be called from interrupt context. Also called by * ib_req_notify_cq() in the generic verbs code. */ -int ipath_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify notify) +int ipath_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags notify_flags) { struct ipath_cq *cq = to_icq(ibcq); unsigned long flags; + int ret = 0; spin_lock_irqsave(&cq->lock, flags); /* @@ -324,9 +325,15 @@ int ipath_req_notify_cq(struct ib_cq *ib * any other transitions (see C11-31 and C11-32 in ch. 11.4.2.2). */ if (cq->notify != IB_CQ_NEXT_COMP) - cq->notify = notify; + cq->notify = notify_flags & IB_CQ_SOLICITED_MASK; + + if ((notify_flags & IB_CQ_REPORT_MISSED_EVENTS) && + cq->queue->head != cq->queue->tail) + ret = 1; + spin_unlock_irqrestore(&cq->lock, flags); - return 0; + + return ret; } /** diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.h b/drivers/infiniband/hw/ipath/ipath_verbs.h index 8039f6e..b0ea548 100644 --- a/drivers/infiniband/hw/ipath/ipath_verbs.h +++ b/drivers/infiniband/hw/ipath/ipath_verbs.h @@ -716,7 +716,7 @@ struct ib_cq *ipath_create_cq(struct ib_ int ipath_destroy_cq(struct ib_cq *ibcq); -int ipath_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify notify); +int ipath_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags notify_flags); int ipath_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata); diff --git a/drivers/infiniband/hw/mthca/mthca_av.c b/drivers/infiniband/hw/mthca/mthca_av.c index 6959945..57cdc1b 100644 --- a/drivers/infiniband/hw/mthca/mthca_av.c +++ b/drivers/infiniband/hw/mthca/mthca_av.c @@ -33,7 +33,6 @@ * $Id: mthca_av.c 1349 2004-12-16 21:09:43Z roland $ */ -#include #include #include @@ -323,7 +322,7 @@ int mthca_ah_query(struct ib_ah *ibah, s return 0; } -int __devinit mthca_init_av_table(struct mthca_dev *dev) +int mthca_init_av_table(struct mthca_dev *dev) { int err; diff --git a/drivers/infiniband/hw/mthca/mthca_cq.c b/drivers/infiniband/hw/mthca/mthca_cq.c index 149b369..5eb5a4d 100644 --- a/drivers/infiniband/hw/mthca/mthca_cq.c +++ b/drivers/infiniband/hw/mthca/mthca_cq.c @@ -36,7 +36,6 @@ * $Id: mthca_cq.c 1369 2004-12-20 16:17:07Z roland $ */ -#include #include #include @@ -723,11 +722,12 @@ repoll: return err == 0 || err == -EAGAIN ? npolled : err; } -int mthca_tavor_arm_cq(struct ib_cq *cq, enum ib_cq_notify notify) +int mthca_tavor_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags) { __be32 doorbell[2]; - doorbell[0] = cpu_to_be32((notify == IB_CQ_SOLICITED ? + doorbell[0] = cpu_to_be32(((flags & IB_CQ_SOLICITED_MASK) == + IB_CQ_SOLICITED ? MTHCA_TAVOR_CQ_DB_REQ_NOT_SOL : MTHCA_TAVOR_CQ_DB_REQ_NOT) | to_mcq(cq)->cqn); @@ -740,7 +740,7 @@ int mthca_tavor_arm_cq(struct ib_cq *cq, return 0; } -int mthca_arbel_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify notify) +int mthca_arbel_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags) { struct mthca_cq *cq = to_mcq(ibcq); __be32 doorbell[2]; @@ -752,7 +752,8 @@ int mthca_arbel_arm_cq(struct ib_cq *ibc doorbell[0] = ci; doorbell[1] = cpu_to_be32((cq->cqn << 8) | (2 << 5) | (sn << 3) | - (notify == IB_CQ_SOLICITED ? 1 : 2)); + ((flags & IB_CQ_SOLICITED_MASK) == + IB_CQ_SOLICITED ? 1 : 2)); mthca_write_db_rec(doorbell, cq->arm_db); @@ -763,7 +764,7 @@ int mthca_arbel_arm_cq(struct ib_cq *ibc wmb(); doorbell[0] = cpu_to_be32((sn << 28) | - (notify == IB_CQ_SOLICITED ? + (flags == IB_CQ_SOLICITED ? MTHCA_ARBEL_CQ_DB_REQ_NOT_SOL : MTHCA_ARBEL_CQ_DB_REQ_NOT) | cq->cqn); @@ -970,7 +971,7 @@ void mthca_free_cq(struct mthca_dev *dev mthca_free_mailbox(dev, mailbox); } -int __devinit mthca_init_cq_table(struct mthca_dev *dev) +int mthca_init_cq_table(struct mthca_dev *dev) { int err; diff --git a/drivers/infiniband/hw/mthca/mthca_dev.h b/drivers/infiniband/hw/mthca/mthca_dev.h index fe5cecf..5da2807 100644 --- a/drivers/infiniband/hw/mthca/mthca_dev.h +++ b/drivers/infiniband/hw/mthca/mthca_dev.h @@ -493,8 +493,8 @@ void mthca_unmap_eq_icm(struct mthca_dev int mthca_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *entry); -int mthca_tavor_arm_cq(struct ib_cq *cq, enum ib_cq_notify notify); -int mthca_arbel_arm_cq(struct ib_cq *cq, enum ib_cq_notify notify); +int mthca_tavor_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); +int mthca_arbel_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); int mthca_init_cq(struct mthca_dev *dev, int nent, struct mthca_ucontext *ctx, u32 pdn, struct mthca_cq *cq); diff --git a/drivers/infiniband/hw/mthca/mthca_eq.c b/drivers/infiniband/hw/mthca/mthca_eq.c index e284e06..8ec9fa1 100644 --- a/drivers/infiniband/hw/mthca/mthca_eq.c +++ b/drivers/infiniband/hw/mthca/mthca_eq.c @@ -33,7 +33,6 @@ * $Id: mthca_eq.c 1382 2004-12-24 02:21:02Z roland $ */ -#include #include #include #include @@ -479,10 +478,10 @@ static irqreturn_t mthca_arbel_msi_x_int return IRQ_HANDLED; } -static int __devinit mthca_create_eq(struct mthca_dev *dev, - int nent, - u8 intr, - struct mthca_eq *eq) +static int mthca_create_eq(struct mthca_dev *dev, + int nent, + u8 intr, + struct mthca_eq *eq) { int npages; u64 *dma_list = NULL; @@ -664,9 +663,9 @@ static void mthca_free_irqs(struct mthca dev->eq_table.eq + i); } -static int __devinit mthca_map_reg(struct mthca_dev *dev, - unsigned long offset, unsigned long size, - void __iomem **map) +static int mthca_map_reg(struct mthca_dev *dev, + unsigned long offset, unsigned long size, + void __iomem **map) { unsigned long base = pci_resource_start(dev->pdev, 0); @@ -691,7 +690,7 @@ static void mthca_unmap_reg(struct mthca iounmap(map); } -static int __devinit mthca_map_eq_regs(struct mthca_dev *dev) +static int mthca_map_eq_regs(struct mthca_dev *dev) { if (mthca_is_memfree(dev)) { /* @@ -781,7 +780,7 @@ static void mthca_unmap_eq_regs(struct m } } -int __devinit mthca_map_eq_icm(struct mthca_dev *dev, u64 icm_virt) +int mthca_map_eq_icm(struct mthca_dev *dev, u64 icm_virt) { int ret; u8 status; @@ -825,7 +824,7 @@ void mthca_unmap_eq_icm(struct mthca_dev __free_page(dev->eq_table.icm_page); } -int __devinit mthca_init_eq_table(struct mthca_dev *dev) +int mthca_init_eq_table(struct mthca_dev *dev) { int err; u8 status; diff --git a/drivers/infiniband/hw/mthca/mthca_mad.c b/drivers/infiniband/hw/mthca/mthca_mad.c index 45e106f..acfa41d 100644 --- a/drivers/infiniband/hw/mthca/mthca_mad.c +++ b/drivers/infiniband/hw/mthca/mthca_mad.c @@ -317,7 +317,7 @@ err: return ret; } -void __devexit mthca_free_agents(struct mthca_dev *dev) +void mthca_free_agents(struct mthca_dev *dev) { struct ib_mad_agent *agent; int p, q; diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c index 47ea021..0491ec7 100644 --- a/drivers/infiniband/hw/mthca/mthca_main.c +++ b/drivers/infiniband/hw/mthca/mthca_main.c @@ -98,7 +98,7 @@ static struct mthca_profile default_prof .uarc_size = 1 << 18, /* Arbel only */ }; -static int __devinit mthca_tune_pci(struct mthca_dev *mdev) +static int mthca_tune_pci(struct mthca_dev *mdev) { int cap; u16 val; @@ -143,7 +143,7 @@ static int __devinit mthca_tune_pci(stru return 0; } -static int __devinit mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) +static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) { int err; u8 status; @@ -255,7 +255,7 @@ static int __devinit mthca_dev_lim(struc return 0; } -static int __devinit mthca_init_tavor(struct mthca_dev *mdev) +static int mthca_init_tavor(struct mthca_dev *mdev) { u8 status; int err; @@ -333,7 +333,7 @@ err_disable: return err; } -static int __devinit mthca_load_fw(struct mthca_dev *mdev) +static int mthca_load_fw(struct mthca_dev *mdev) { u8 status; int err; @@ -379,10 +379,10 @@ err_free: return err; } -static int __devinit mthca_init_icm(struct mthca_dev *mdev, - struct mthca_dev_lim *dev_lim, - struct mthca_init_hca_param *init_hca, - u64 icm_size) +static int mthca_init_icm(struct mthca_dev *mdev, + struct mthca_dev_lim *dev_lim, + struct mthca_init_hca_param *init_hca, + u64 icm_size) { u64 aux_pages; u8 status; @@ -575,7 +575,7 @@ static void mthca_free_icms(struct mthca mthca_free_icm(mdev, mdev->fw.arbel.aux_icm); } -static int __devinit mthca_init_arbel(struct mthca_dev *mdev) +static int mthca_init_arbel(struct mthca_dev *mdev) { struct mthca_dev_lim dev_lim; struct mthca_profile profile; @@ -683,7 +683,7 @@ static void mthca_close_hca(struct mthca mthca_SYS_DIS(mdev, &status); } -static int __devinit mthca_init_hca(struct mthca_dev *mdev) +static int mthca_init_hca(struct mthca_dev *mdev) { u8 status; int err; @@ -720,7 +720,7 @@ err_close: return err; } -static int __devinit mthca_setup_hca(struct mthca_dev *dev) +static int mthca_setup_hca(struct mthca_dev *dev) { int err; u8 status; @@ -875,8 +875,7 @@ err_uar_table_free: return err; } -static int __devinit mthca_request_regions(struct pci_dev *pdev, - int ddr_hidden) +static int mthca_request_regions(struct pci_dev *pdev, int ddr_hidden) { int err; @@ -928,7 +927,7 @@ static void mthca_release_regions(struct MTHCA_HCR_SIZE); } -static int __devinit mthca_enable_msi_x(struct mthca_dev *mdev) +static int mthca_enable_msi_x(struct mthca_dev *mdev) { struct msix_entry entries[3]; int err; @@ -1213,7 +1212,7 @@ int __mthca_restart_one(struct pci_dev * } static int __devinit mthca_init_one(struct pci_dev *pdev, - const struct pci_device_id *id) + const struct pci_device_id *id) { static int mthca_version_printed = 0; int ret; diff --git a/drivers/infiniband/hw/mthca/mthca_mcg.c b/drivers/infiniband/hw/mthca/mthca_mcg.c index 47ca8a9..a8ad072 100644 --- a/drivers/infiniband/hw/mthca/mthca_mcg.c +++ b/drivers/infiniband/hw/mthca/mthca_mcg.c @@ -32,7 +32,6 @@ * $Id: mthca_mcg.c 1349 2004-12-16 21:09:43Z roland $ */ -#include #include #include @@ -371,7 +370,7 @@ int mthca_multicast_detach(struct ib_qp return err; } -int __devinit mthca_init_mcg_table(struct mthca_dev *dev) +int mthca_init_mcg_table(struct mthca_dev *dev) { int err; int table_size = dev->limits.num_mgms + dev->limits.num_amgms; diff --git a/drivers/infiniband/hw/mthca/mthca_mr.c b/drivers/infiniband/hw/mthca/mthca_mr.c index a486dec..f71ffa8 100644 --- a/drivers/infiniband/hw/mthca/mthca_mr.c +++ b/drivers/infiniband/hw/mthca/mthca_mr.c @@ -34,7 +34,6 @@ */ #include -#include #include #include "mthca_dev.h" @@ -135,7 +134,7 @@ static void mthca_buddy_free(struct mthc spin_unlock(&buddy->lock); } -static int __devinit mthca_buddy_init(struct mthca_buddy *buddy, int max_order) +static int mthca_buddy_init(struct mthca_buddy *buddy, int max_order) { int i, s; @@ -759,7 +758,7 @@ void mthca_arbel_fmr_unmap(struct mthca_ *(u8 *) fmr->mem.arbel.mpt = MTHCA_MPT_STATUS_SW; } -int __devinit mthca_init_mr_table(struct mthca_dev *dev) +int mthca_init_mr_table(struct mthca_dev *dev) { unsigned long addr; int err, i; diff --git a/drivers/infiniband/hw/mthca/mthca_pd.c b/drivers/infiniband/hw/mthca/mthca_pd.c index 59df516..c1e9507 100644 --- a/drivers/infiniband/hw/mthca/mthca_pd.c +++ b/drivers/infiniband/hw/mthca/mthca_pd.c @@ -34,7 +34,6 @@ * $Id: mthca_pd.c 1349 2004-12-16 21:09:43Z roland $ */ -#include #include #include "mthca_dev.h" @@ -69,7 +68,7 @@ void mthca_pd_free(struct mthca_dev *dev mthca_free(&dev->pd_table.alloc, pd->pd_num); } -int __devinit mthca_init_pd_table(struct mthca_dev *dev) +int mthca_init_pd_table(struct mthca_dev *dev) { return mthca_alloc_init(&dev->pd_table.alloc, dev->limits.num_pds, diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index fc67f78..21422a3 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -1100,11 +1100,10 @@ static struct ib_fmr *mthca_alloc_fmr(st struct mthca_fmr *fmr; int err; - fmr = kmalloc(sizeof *fmr, GFP_KERNEL); + fmr = kmemdup(fmr_attr, sizeof *fmr, GFP_KERNEL); if (!fmr) return ERR_PTR(-ENOMEM); - memcpy(&fmr->attr, fmr_attr, sizeof *fmr_attr); err = mthca_fmr_alloc(to_mdev(pd->device), to_mpd(pd)->pd_num, convert_access(mr_access_flags), fmr); diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index 6a7822e..33e3ba7 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -35,7 +35,6 @@ * $Id: mthca_qp.c 1355 2004-12-17 15:23:43Z roland $ */ -#include #include #include @@ -2241,7 +2240,7 @@ void mthca_free_err_wqe(struct mthca_dev *new_wqe = 0; } -int __devinit mthca_init_qp_table(struct mthca_dev *dev) +int mthca_init_qp_table(struct mthca_dev *dev) { int err; u8 status; diff --git a/drivers/infiniband/hw/mthca/mthca_srq.c b/drivers/infiniband/hw/mthca/mthca_srq.c index f5d7677..58fcf5a 100644 --- a/drivers/infiniband/hw/mthca/mthca_srq.c +++ b/drivers/infiniband/hw/mthca/mthca_srq.c @@ -715,7 +715,7 @@ int mthca_max_srq_sge(struct mthca_dev * sizeof (struct mthca_data_seg)); } -int __devinit mthca_init_srq_table(struct mthca_dev *dev) +int mthca_init_srq_table(struct mthca_dev *dev) { int err; diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 0b8a79d..025cef2 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -239,6 +239,7 @@ extern struct workqueue_struct *ipoib_wo /* functions */ +int ipoib_poll(struct net_device *dev, int *budget); void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr); struct ipoib_ah *ipoib_create_ah(struct net_device *dev, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 8bf5e9e..7c0fe8f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -286,26 +286,58 @@ static void ipoib_ib_handle_tx_wc(struct wc->status, wr_id, wc->vendor_err); } -static void ipoib_ib_handle_wc(struct net_device *dev, struct ib_wc *wc) +int ipoib_poll(struct net_device *dev, int *budget) { - if (wc->wr_id & IPOIB_OP_RECV) - ipoib_ib_handle_rx_wc(dev, wc); - else - ipoib_ib_handle_tx_wc(dev, wc); + struct ipoib_dev_priv *priv = netdev_priv(dev); + int max = min(*budget, dev->quota); + int done; + int t; + int empty; + int n, i; + +repoll: + done = 0; + empty = 0; + + while (max) { + t = min(IPOIB_NUM_WC, max); + n = ib_poll_cq(priv->cq, t, priv->ibwc); + + for (i = 0; i < n; ++i) { + if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) { + ++done; + --max; + ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); + } else + ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); + } + + if (n != t) { + empty = 1; + break; + } + } + + dev->quota -= done; + *budget -= done; + + if (empty) { + netif_rx_complete(dev); + if (unlikely(ib_req_notify_cq(priv->cq, + IB_CQ_NEXT_COMP | + IB_CQ_REPORT_MISSED_EVENTS)) && + netif_rx_reschedule(dev, 0)) + goto repoll; + + return 0; + } + + return 1; } void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) { - struct net_device *dev = (struct net_device *) dev_ptr; - struct ipoib_dev_priv *priv = netdev_priv(dev); - int n, i; - - ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); - do { - n = ib_poll_cq(cq, IPOIB_NUM_WC, priv->ibwc); - for (i = 0; i < n; ++i) - ipoib_ib_handle_wc(dev, priv->ibwc + i); - } while (n == IPOIB_NUM_WC); + netif_rx_schedule(dev_ptr); } static inline int post_send(struct ipoib_dev_priv *priv, @@ -510,9 +542,10 @@ int ipoib_ib_dev_stop(struct net_device struct ib_qp_attr qp_attr; unsigned long begin; struct ipoib_tx_buf *tx_req; - int i; + int i, n; clear_bit(IPOIB_FLAG_INITIALIZED, &priv->flags); + netif_poll_disable(dev); /* * Move our QP to the error state and then reinitialize in @@ -559,6 +592,16 @@ int ipoib_ib_dev_stop(struct net_device goto timeout; } + do { + n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); + for (i = 0; i < n; ++i) { + if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) + ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); + else + ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); + } + } while (n == IPOIB_NUM_WC); + msleep(1); } @@ -587,6 +630,9 @@ timeout: msleep(1); } + netif_poll_enable(dev); + ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP); + return 0; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 85522da..4a63e00 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -895,6 +895,8 @@ static void ipoib_setup(struct net_devic dev->hard_header = ipoib_hard_header; dev->set_multicast_list = ipoib_set_mcast_list; dev->neigh_setup = ipoib_neigh_setup_dev; + dev->poll = ipoib_poll; + dev->weight = 100; dev->watchdog_timeo = HZ; diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 4b09147..613be2e 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -1716,7 +1716,8 @@ static ssize_t srp_create_target(struct if (!target_host) return -ENOMEM; - target_host->max_lun = SRP_MAX_LUN; + target_host->max_lun = SRP_MAX_LUN; + target_host->max_cmd_len = sizeof ((struct srp_cmd *) (void *) 0L)->cdb; target = host_to_target(target_host); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 8eacc35..42f59f4 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -428,9 +428,11 @@ struct ib_wc { u8 port_num; /* valid only for DR SMPs on switches */ }; -enum ib_cq_notify { - IB_CQ_SOLICITED, - IB_CQ_NEXT_COMP +enum ib_cq_notify_flags { + IB_CQ_SOLICITED = 1 << 0, + IB_CQ_NEXT_COMP = 1 << 1, + IB_CQ_SOLICITED_MASK = IB_CQ_SOLICITED | IB_CQ_NEXT_COMP, + IB_CQ_REPORT_MISSED_EVENTS = 1 << 2, }; enum ib_srq_attr_mask { @@ -941,7 +943,7 @@ struct ib_device { struct ib_wc *wc); int (*peek_cq)(struct ib_cq *cq, int wc_cnt); int (*req_notify_cq)(struct ib_cq *cq, - enum ib_cq_notify cq_notify); + enum ib_cq_notify_flags flags); int (*req_ncomp_notif)(struct ib_cq *cq, int wc_cnt); struct ib_mr * (*get_dma_mr)(struct ib_pd *pd, @@ -1366,14 +1368,34 @@ int ib_peek_cq(struct ib_cq *cq, int wc_ /** * ib_req_notify_cq - Request completion notification on a CQ. * @cq: The CQ to generate an event for. - * @cq_notify: If set to %IB_CQ_SOLICITED, completion notification will - * occur on the next solicited event. If set to %IB_CQ_NEXT_COMP, - * notification will occur on the next completion. + * @flags: + * Must contain exactly one of %IB_CQ_SOLICITED or %IB_CQ_NEXT_COMP + * to request an event on the next solicited event or next work + * completion at any type, respectively. %IB_CQ_REPORT_MISSED_EVENTS + * may also be |ed in to request a hint about missed events, as + * described below. + * + * Return Value: + * < 0 means an error occurred while requesting notification + * == 0 means notification was requested successfully, and if + * IB_CQ_REPORT_MISSED_EVENTS was passed in, then no events + * were missed and it is safe to wait for another event. In + * this case is it guaranteed that any work completions added + * to the CQ since the last CQ poll will trigger a completion + * notification event. + * > 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was passed + * in. It means that the consumer must poll the CQ again to + * make sure it is empty to avoid missing an event because of a + * race between requesting notification and an entry being + * added to the CQ. This return value means it is possible + * (but not guaranteed) that a work completion has been added + * to the CQ since the last poll without triggering a + * completion notification event. */ static inline int ib_req_notify_cq(struct ib_cq *cq, - enum ib_cq_notify cq_notify) + enum ib_cq_notify_flags flags) { - return cq->device->req_notify_cq(cq, cq_notify); + return cq->device->req_notify_cq(cq, flags); } /**